DOCUMENT RESUME 



ED 304 352 



SO 019 569 



AUTHOR 
TITLE 



INSTITUTION 

SPONS AGENCY 

PUB DATE 

GRANT 

NOTE 

PUB TYPE 



Newmann, Fred M.; And Others 

Higher Order Thinking in High School Social Studies: 
An Analysis of Classrooms, Teachers, Students, and 
Leadership. 

National Center on Effective Secondary Schools, 
Madison,. WI. 

Office of Educational Research and Improvement (ED) , 

Washington, DC. 

Jul 88 

G-008690007 

222p. 

Reports - Research/Technical (143) — Collected Works 
- General (020) 



EDRS PRICE 
DESCRIPTORS 



i:d5Ntifiers 



MF01/PC09 Plus Postage. 

Cognitive Development; *Cognitive Processes; Critical 
Thinking; High Schools; *High School Students; 
Research Projects; *Secondary School Curriculum; 
*Secondary School Teachers; *Social Studies 
*Higher Order Skills; *Thinking Skills 



ABSTRACT 

The papers in this report present selected findings 
from one phase of a five-year study on how to enhance higher order 
thinking skills in high school social studies classes. The study's 
purposes were to determine why it is difficult to emphasize these 
skills in sccial studies curriculums, what the barriers are, and how 
to overcome them. Five demographically diverse high schools were 
selected for the study. The results provided in six chapters include: 
(1) "The curriculum of Thoughtful Classes" (F. Newmann) ; (2) 
"Exploring Aspects of Teachers* Thinking about Promoting Students' 
Thinking" (J. Onosko); (3) "Classroom Practices of High and Lower 
Scoring Teachers" (J. Onosko); (4) "Student Perspectives on 
Cognitively Challenging Curriculum" (R. Stevenson); (5) "Student 
Perspectives on Engaging Curriculum" (R. Stevenson) ; and (6) 
"Departmental and Principal Leadership in ProiP'-ting Higher Order 
Thinking" (C. McCarthy; F. Schrag) . Findings indicate that classroom 
thoughtfulness in social studies can be assessed and can occur at 
high levels in conventionally organized high schools among students 
of all grade and achievement levels. Tables and references are 
included. (JHP) 
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I Outline of the I^joger Stut^ 

Hie papers in this report present selected finding from one phase 
of a five-year study of how to enhance hi^ier order thinking in hi^ 
school social studies. Based on persii^tent cledins frm previous 
research that hi^ier order thinking is rarely observed in hi^ school, 
the larger study asks two central questiwis: (A) Why is it apparently 
so difficult to eiiphasize hi^er order thinking in the curricAilum- 
\ihBt are the key barriers? and (B) To vdiat extent is it pc^ible for 
hit^ school social studies departaients to cverccane the barriers and 
\*at is required to do so? 

Ihe full study inclvides syntheses of research related to the 
conceptualization of thinking (Newmann, in press) , ado"^.escent 
capacities for hi^er order thou^t (Ifeating, 1988) , and organi.zational 
constraints on the prcffaotion of thinking (Marrett and Kim, 1987) . 
Enpirical work will ccorrpare the degree of hi^ier order thinking, 
barriers, and responses to barriers in three different sets of social 
studies departments: (a) those that place special eitjiiasis on ai^er 
order thirSdng, but that organize instruction accordii^ to faniliar 
patterns in the cortprehensive hi^ school; (b) those that itcto no 
special departmental-wide efforts and are conventionally organized; and 
(c) those that involve a de>artinental eirphasis on hi^ier order thinking 
and have also made significant changes in the organization of 
instruction (as suggested by ^he term "restructured" schools) . 

Selection of deparbrents, teachers and lessons was guided by the 
larger project's effort to identify "exertplary" social studies 
departments (that is, those that make a serious departrafintal-wide 
effort to enjjhasize hi^ier order thinking) and then, by a)ntrasting 
these d^)artments with others, to discover how scate d^)artiaents may be 
able to overcome barriers that others cannot. Tlius, rather than 
concentrating primarily upon differences between individual teachers, 
this study e^lores the pr-±)lem of institutionalization -^t is 
required for d^>artmental-wide prcanotion of hi^ier order thinking? 

Ihis r^rt focuses only on the study of select departLients 
mentioned as type (a) above. Ihe papers all rely iipon a ocmrtm 
conception of hi^ier order thinking summarized in Newmann's paper, and 
the data for all papers was gathered according to methodology described 
below. 



n Msfthodology fear the Stu^ of Select D^rtirypnts 



Selection of Social Studies Depa2±ments , Based on a national 
search, five dqpartinents from deinograjSiically diverse hi^ schools were 
selected for stucty. Criteria eitphasized a d^>artinental--wide effort to 
pranote thinking for all students, not only for hi^ achievers. 

Departments were selected throu^ announcements in professional 
publications and special mailingr to persons in the social studies 
ccrarromity \^ could help to ncoodnate schools. Several criteria were 
eniphasized in these annoancemsnts: a deparbrvental ercphasis on hi^er 
order thinking; a required course in social studies, preferably US 
History, that en?iiasizes the study of topics in depth; the presentation 
of conteiTt as problematic, changing or controversial, rather than 
authoritative, fixed and true; students involved in problem solving and 
interpreting information; students reasonir^ about their views and 
receiving feedback from the teacher on the quality of their reasoning. 

Sixty ncminations were received, followed by correspondence, phone 
interviews, and a one-day site visit to nine schools. D^)artiaents in 
five of these schools were selected for stody. To maximize candor in 
interviews aixJ to protect subjects, confidentially and anonymity was 
premised to all individuals. Therefore, the names of schools and 
individuals in this r^rt are fictitious. As illixstrated in Table 1, 
the sample included large and medium-^ized schools, schools in both 
single- and multi- hi^ school districts, urban and suburban schools, 
and schools with a significant proportion of minority and disadvantaged 
students. Notably absent from this small set of schools was a small 
rural hi^ school or a school with a drop out rate as large as the 
estimated national average of 25%. 

Class (&servatiaTs. The department chair at each sctiool selected 
three main courses, tau^t by different teachers, to be cbserved nine 
times over three visits to the school in 1986-1987. The thr^ classes 
were to illustrate as mich hi^er order thinking as possible, but they 
were to include (a) a class with a substantial proportion of laver and 
middle adiieving students; (b) a history course with a diverse range 
of students ; and (c) any other class that best illustrates a 
concentration on hi<^ier order thinking (vMch usually cortprised hi^ 
achievers) . On each visit, two additional classes, also presumed to 
eicfiiasize hi^er order thinking and tau^t by different teachers, were 
deserved once. - This would result in a total of 165 lessons, but due to 
absences or bad weather, the final sanple of lessons was 160. Within 
scheduling consitraints, teachers were encouraged to select for our 
observation those lessons that placed most enptiasis on hi^ier order 
thinking. 

(S^servations were recorded on seventeen 5-point scales reflecting 
dimensions of thou^tful discourse (e.g. Were students challenged to go 
beyond the infontation given? Were reasons carefully considered?) . 
Descriptive notes elaborated on specific practices and responses. 
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Tasks assigned by the teacher that required writing were also collected 
and evalviated, as was written work by three students in each class. 

Teacher Qaestionnaires and interviews. The three teachers per 
school vdiose classes were observed nine tiines were designated primary 
teachers, and they ocatpleted three questionnaires and at least six 
hours of ^ interview dealing with classroom observation and their views 
on a variety of issues related to the prcciotion of thinldng. Tteachers 
v.ho were observed only ciyx constituted the secondary sanple, and they 
ccirpleted two questionnaires and one hour of interview. Ttotal teachers 
~ 45 (15 primary, 30 secondary) . 

Students. Three students from each of the 15 primary classes 
ccaipleted one hour interviews during each of three school visits and an 
total of two questionnaires. Teachers selected students r^resenting 
the hi^iest, middle, and Icwest 1/3 of achievement in the class. 
Student data focused on engagement in schoolwork, their reactions to 
observed lessons and to hi^ier order thinking tasks. Ototal students = 
45. 

The social studies department chair and principal ccgrpleted 
questionnaires and a minimum of three one-hour interviews dealing with 
how the deparbnent and school organization promotes hi^ier order 
thinking. 

HI Overviesj of the Papers 

This study assumes that instead of trying to discover discrete 
teaching techniques to teach particular thirikir^ skills to individual 
students, researtii in hi^ school social studies should first take a 
broader perspective and examine how to promote more general qualities 
of thou^tfUlness in classroom discourse. This research entails 
developing a method for assessing classroom thou^tfuiness and also 
studying the views of teachers, administrators and students to learn 
more about barriers to its promotion and how they mi^t be overxxme. 

Newnann develops the rationale for the study of classroom 
thou^tfulness and explains the conservation scheme of 5~point scales. 
After presenting findings on the amount of variation within and between 
d^)artments, he describes teacher and student behaviors associated with 
lessons that score hi^ versus lew an thoi^tfulness. He conclxides by 
examining the possible effect on classroom thoucjhtfulness of students' 
age, school achievement level and miriority status. 

Newmann's analysis of the lessons as a v^le is caiplemented by 
Qnosko's study of teadiers. In v^iat ways migjit teachers • beliefs about 
teaching influence the degree of thou^tfulness »:rcmoted in their 
lessons? ^ Qnosko oarpares the beliefs of five teachers whose lessons 
are consistently rated hi^ on thcu^.tfulmss v;ith five teachers viiose 
lessons receive lower ratings* He describes how the teachers differ in 
the instr\x±ional goals they eitfJiasize, the ccaotplexilg^ of their 
rationales, their views on the issue of breadth vs depth in covering 
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subject natter, their perwiptions of barriers to prorooting hi^er order 
thinking, and their peraiptions factors that affect hi^ and low 
achieving students' thinking, including the teacher's sense of 
influence over these factors. The second paper by Onosko describes 
specific ways in vAiich hic^er and lower scoring teachers differ in 
their classroan practices and the ejctent to vAiich these differences 
persist across all lessons. 

Do students pose a major barrier to hi^er order thinking in 
school by preferring low level cognitive work? Stevenson's first paper 
describes the kinds _ of instructional activities that students find 
cognitively challenging. He discusses student perceptions of social 
studies in relation to other subjects and the kinds of tasks and nvental 
processes that they consider most difficult, for example, their 
reactions to memorization, conceptual learning, making inferences and 
taking positions. Ihe second paper by Stevenson describes activities 
that students consider engaging; that is, those that they are willing 
to work hard at and that they find interesting and worthwhile. 
a3ogether, Stevenson's two papers offer information on the extent to 
viiidi students find social studies in these select departments both 
engaging and mentally challenging. 

While each of the departments was initially selected eis exenplary 
for its apparent eitphasis on hi^er order thinking, we found 
considerable variation among the five schools, with three hi^ 
performing d^artments clearly separated from two lower scoring ones. 
How mi^t the differences between schools ejqjlained? In studying 
interviews of department chairs and principals, McCarthy and Schrag 
examine the extent of programmatic differences, the level of resources, 
the nature of collegial relations, and most iitportantly the qualities 
of departmental and principal leadership that may affect the 
d^)artmental level of classroom thouc^tfulness. Analysis of this sort 
may lead ultimately to reccmmendations for school and district policies 
to minimize the barriers and maximize opportunities for hi^er order 
thinking. 

This report coitpletes some initial steps in a longer journey, but 
several tasks lie ahead before it will be possible to answer the main 
questions of the larger study. For thj.s reason, the conclusion offers 
only a brief summary of the major findings of this phase in the 
research and an outline of analyses anticipated in the future. 
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Table 1 
Sc3x»l Denogrc?!hic Profiles 



1. 1986 Enrollinent 

2. Ethnic Racial Ccnposition 

i % White 

ii % Black 

iii % Asian 
iv % Hispanic 

V % other Minority 

3. % Lew Inccana 

4. Number of Teachers 

5. % 1986 Graduates Going To: 

i 4 Year Oollego 

li Technical School 

iii 2 Year Cccmainity colleae 

iv Military 

V Job + Military 

6. 1986 Percent Droj^-out Rate, Based on 
4 Years 

7. Per Pt^il E>?)enditure 1986 



A 



1^6? 

65.9 
2.8 
14.1 
12.6 
4.6 

3 

72 



40 
15 
30 
3 
12 



SCHOOL* 




„ B 


C 


6 


S 


2140 


2830 


2022 


1089 


75 


49,9 


91 


32 


9 


22.2 


3 


67 


12 


20.4 


3 


1 


4 


7.3 


3 


0 


0 


.1 


0 


0 


7 


16 


<1 


42 


176 


145 


185. 6^ 


91 



72 
2 
6 
2 

18 



3.39 9.3 

3395 4100 



60 
1.5 
30.7 
1.9 
3.8 



69 
4 
5 
1 

14 



16.8 1 
4587 4600 



37 
14 
17 
10 
22 



19 
5839 



°f ^® schools; Arnold, Bradley, 
Carlsberg, Grandville, Scarborough ^^^'^^Y' 



^Includes specialists 
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The corriculm of !IS>au^itful Classes 
Fred M. Neraam 

To ujiderstand the difficulties and possibilities of prcaioting 
hi^oer order thinking, it is necessary to build both a conceptual 
fcundation and eirpirical procedures that allow vis to describe the 
extent to which hic^er order thinking occurs in classroons. Relying on 
the concQjtion of hi^er order thinking e^qplained by Newirann (in 
press) , this paper presents a rationale for the assessment of classrocm 
thoo^tfulness, an instrument and procedures for assessing it in 
classrocros, and findings on classrocm practices and characteristics of 
schools that may account for differences between the roost and least 
thou^tful lessons, 

I Rationale for the Assessanent of CLassroam Ihou^itfulness 

In previous work that reviews literature on the nature of thinking 
(Newmann, in press) we have presented a conception of hi^ier order 
thirikir^ applicable to teachirg a variety of subjects. Hic^ier order 
thinking is defined as a diallei^e that requires the person to go 
beyond the ^ infonration given; that is, to interpret, analyze, or 
manipulate information, because a question to be answered or a problem 
to be solved cannot be resolved throu^ the routine application of 
previously learned knowledge* m contrast, lower order thinJdrg 
demands only routine, mechanistic application of previously acquired 
knowledge; for exaitple, repetitive exercises sudi as listing memorized 
information or inserting numbers into mathematical formulas. Tasks 
that call for non-routine mental work have been considered a 
distinguishing feature of hi^er level thinkir^ in reviews of 
literature on the topic in psychology, cognitive science, education, 
and philosophy (Patterson and Smith, 1986; Resnick, 1987; Schrag, in 
press) . 

Siitply presenting students with tasks that challer^e them to go 
beyond the informaticxi given will not necessarily nurture success in 
meeting the tasks. What is needed to be successful with hic^ier order 
ciiallenges? We argue that in order to meet hi^er order challenges 
successfully, students need a ccxiibination of in-depth knowledge, skills 
in tising knowledge, and, most xitportantly, dispositions or attitudes of 
thought fulness. Ihese dimensions help to describe differences between 
people ^fA)o succeed and fail in hi^er order tasks, and they suggest, 
therefore, three different octtponents that ocu^t to be taken into 
account vAien planning curriculum and instruction to proiote thinking. 
MUch of the current programmatic eitjiiasis to pranote better thinking 
concentrates mainly on the development of skills. A focus on skills 
that neglects knowledge of specific subject matter and cultivation of 
dispositions of thou^tfulness, however, is not likely to effect 
substantial, long-run improvement in students' thinking to meet hi^er 
order challenges. 
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Rather than selecting a specific conception of thinking such as 
critical thinking, informal reasoning, inoral reasoning or divergent 
thinking, at this tijna it is 3i>ore prudent to work toward a broad 
conception. Our research with social studies teachars indicates that 
calls for specific types of thinking are lonlikely to generate 
widespread acceptance of any particular type. Hi^ school teachers are 
likely to perpetuate their previous concerns for a plurality of types 
of thinking, but even these will be grounded primarily in the teaching 
of their subjects . A broad conception that embraces diverse subjects 
is Bore liJcely to attract wide practitioner support. 

Along with a broad conception of thinkirg, rtxxsnmend an 
approach to research that avoids fragitventation of knowledge. One mi^t 
envision a long-term research effort to discover all the different ways 
of teaching specific content, skills and dispositions a-z to solve 
particular problems or challenges 1-n. Such balkanization of 
kncwledge, however, would make it ever inore difficult to synthesize 
findings usefully for practitioners. 

Finally, a broad conception of thinking can strike at the heart cf 
an underlying iiala^ identified by mai^ studies of schools. At best, 
mch classrxxsn activity fails to challenge students to use their minds 
in any valuable ways; at worst, much classroccn activity is nonsensical 
or mindless. Ihe more serious problem, therefore, is not the failure 
to teach soie specific aspect of thinkirg, but the profound absence of 
thouc^tfulness in classrocsns (Cuban, 1984; Goodlad, 1984; Perxone, 
1985; Potvell et al, 1985; Stake and Easley, 1978). As suggested above, 
even programs to teach thinking skills can fail to prxxivote 
thoK^tfulness, A general conception of thinking is xcore likely to 
address the more basic issue of the general lack of .intellectual 
challenge, because it will recognize a variety of ways in which 
thought fulness can be prowoted. 

One way to learn how to promote hi^er order thinking is to find 
teachers ^Aio do it rather consistently and, to lock for the ways in 
which th^ differ fran those yAio demonstrate less success. If we can 
distinguish between more and less thou^tful dassroone, we can then 
inquire more systematically about the possible factors that influence 
the praiotion of higher order thinking in classrooms. Do teachers \*o 
prcanote hi^i© order thinking have different goals, conc^ions of 
their subject, relations with colleagues and beliefs about students 
than teachers viho teach in mora routine ways? Do seme departments 
offer specific kinds of leadership, trairujig and si^jport that lead to 
greater departmsnt-wide earphasis on hi^er order thirfdng? Does hi^er 
order thinking ocaur only in those classes with large proportions of 
hig^i achieving students? A first step in answeri7>g these questions is 
to devise a measure or index of classrocm thou^tfulness that 
distinguishes between dassrooms that are more and less effective in 
prorooting hi^er order thinking. 

One way of documenting classroom thou^tfulness is to assess the 
kind of thinking demonstrated by individual students and then to 
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aggregate these measures for the vAiole class. But acxording to our 
definition of higher order thinking, individuals will differ on the 
kinds of problems they find (diallenging. For one person, trying to 
understand hew to read and follow a bus schedule nay require hi^er 
order thou^t, but for another, the saioe task will be routirse. In this 
sense, higher order thinking is relative: to detenaine the extent to 
vMch an individual is facing a hi^er order challenge, one would 
presumably need to knew sooething of the person's intellectual history. 
Further;, to assess the extent to which the person accually participates 
in the analysis, interpretation and manipulation of information, one 
Tvc^d want to "get inside" the person's head or eaqperienoe his/her 
subjective state of thou^t. Hiis, of course, is extremely difficult 
to do, especially with a vdiole class of students. 

Because of these difficulties in assessing the thinking of 
individual students daring their classes, we chose instead to examine 
each lesson as a whole. We may have little opportunity to assess 
students' individual mental states, but we can make assunptions about 
the prior knowledge of groups of students and, therefore, about the 
kinds of mental work that certain tasks are likely to stimulate. This 
knowledge allows us to estimate the extent to ^ch students are 
confronted with non-routine challenges. We can also look for patterns 
in classroon disccurse that reflect hew thou^tfully students and the 
teacher cope with the challenges. That is, we can assess behavior that 
r^resents in-depth understanding of subject content, skills in 
manipulation of information, and dispositions of thou^tfulness. 

This focus on the classroom, rather than the individual student, 
is not only an operational necessity; it also builds directly on a^ 
central finding from several studies of instruction in schools; namely 
the lew level of discourse in classes, the lack of thinking by 
individual students. 

Having presented a conception of hi^ier order thinking and 
explciined the need for assessment at the classrooKi level, we must not 
neglect the more fundamental question: vAiy are higher order thinking 
and classroom thoughtftalness desirable educational goals? There is 
much dispute on the nature of thinking and how to teach or to prtatote 
it, but the common rationales can be summarized as three main 
arguments. Thou^tfulness is necessary for people (a) to participate 
as responsible, eitpowered citizens in a denocracy (Oliver & Shaver, 
1966), (b) to contribute as produ;^ive workers in a techr^logical 
society (Ccffnmit±ee for Economic Development, 1985), and (c) to have 
rewarding personal lives vMcii includes managing one's private affairs, 
continuing to learn and benefitting from culture (Schrag, in press) . 

We cissume that thou^tftil lessons, by presenting students with 
hi^er order challenges and nurturing the knowledge, skills and 
dispositions required to meet the challenges sucoessftdly, will enhance 
these benefits for individual students. This stage of the enpirical 
research does not assess the effects of classroom thou^tfulness on 
individual students, but we would predict that if individual 
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assessments were conducted, students v*o participate in itore thoo^tful 
classrocans will ha:*/e greater success on oral and written assignitents 
that pose hi^er order challenges in subjects closely related to the 
class. Such students are also itore liJcely to succeed on tests of 
sinple information retrieval in the subjects studied. A future phase 
of the research will examine the relationship between classroom 
thcui^tfulness and student achievement. 

Apart firom its effects on individual achievement, hcwever, the 
promotion of classroom thou^tfulness can be defended as worthwhile on 
the ontological claita that xising the mind to confront new challenges is 
critical to human nature (Rawls, 1971). To the extent that we are 
derived of the opportunity to work on these challenges, we are tmable 
to express a central part of our being. 

Having esg>lained the conceptual basis of the stuc^, the ensuing 
r^rt of eirpirical work addresses two madn questions: (A) What 
dimensions of teacher and student discourse can be iised to describe the 
degree of hi^ier order thinking prcomoted in social studies classes? 
(B) What teaching practices, characteristics of schools, and 
characteristics of students tend to distinguish between the most and 
l^ast thou^tful lessons? 

U An InstxiJDioit tx> Assess ClassroceDi 01^^ 

A. Guidelines 

Several considerations guided the development of our instrument. 
Die principal concern wa£ to develop an observation scheme consistent 
with the general conception of hi^ier order thinking outlined above. 
More specific guidelines such the following were also used: 

> Ihe categories or variables should be able to be 

observed in the teaching of a variety of subject matter and skills 
vrfthin social studies. 

> Cie categories should refer to teacher behavior, to student 
behavior, ai^ to activities involving both teacher and student. 

> As an e55>loratory instrument, the categories should inclxxte many 
dimensions vdiioh, on the basis of further theoretical and eitpirical 
analysis, mi^t later be reduced to a smaller number of essential 
criteria. 

> Ohe categories should be conc^jtualized in ways that mi^t later 
be iised to help teachers reflect on their practice. 

Using these guidelines, the research team considered the 
theoretical and practical advantages of a large number of criteria. 
Based on observations of several live and videotaped classes, 
discussions with ejqjerienced teachers, and exercises to achieve inter- 
rater reliability, seventeen observational scales were selected. 
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B. Sie Scales 

Bie scales are described in detail in the i^^jei^dix to this j»per. 
General characteristics of the lesson vihidi are normally the 
responsibility of the teacher include 1, sustained examination of a few 
topics rather than superficial coverage of many, 2. substantive 
coherence and continuity, and 3, sufficient time to think, Ihe absence 
of any of these will, we presume, undermine the disposition to think 
things throu^ systematically and deliberately. 

Several scales focus more specifically on teacher behavior. The 
thou^trftil tieacher must 4 • ask challenging questions or pc^ 
challenging tasks. In guiding students' work with the taslcs, the 
teacher should 5, carefully consider explanations and reasons for 
conclusions, 6. press individual students to justify or clarify their 
assertions, 7. encourage students to generate original and 
xjnconventional ideas, 8. ahcw an awareness that not all assertions 
emanating fecm authoritative sources are absolute and certain, 9. 
integrate students' personal ejqperience (viiere relevant) into the 
lesscxi, and 10. model other characteristics of a thoi^tful person. 

Ihese teacher characteristics will help to reinforce in-d^jth 
understanding and the qualities of reflection characteristic of 
effective thinkers. Conversely their absence undermines student 
success with hi^er order challenges. For exairple, a teacher ;*o does 
not press stxdents to clarify or justify their answers reinforces their 
tendency to say vAiatever comes into their heads without considering its 
warrant or iirplications. Or, a teacher who doesn't give students 
encouragement for legitimate but x:incanventional responses si^jports the 
stereotyped responses vMch characterize weak thinkers* 

Several scales focus on student behavior. In thou^tful 
classrocjns, students 11. offer es^lanations and reasons for their 
conclusions, 12. generate original and unconventional ideas, 13. assume 
the role of questioner and critic, airi 14. make contributions vMch are 
articulate and germane to the topic. To behave in these ways, students 
must participate verbally (15, 16). If these behaviors are not 
evident, other cues may indicate engagement in mental work (17) . 

Based on criteria \Mch will be ejqplained later, six of the scales 
were eventxially selected as minimal indicators of classroom 
thou^t fulness . 



Our basis for determining thou^tfulness was verbal 
discourse dealing with the content of lessons, but indicators of 
thcaa^tfulness in other areas mi^t also have been examined; for 
ejcairple, rules for student behavior and handling of violations; 
grading practices ; assigning stud^ responsibility for non- 
instructional tasks such as classroom clean-i^) and distributing 
materials. Thou^tful discourse in the formal part of lessons may 
be related to these aspects of classroom management. These areas 
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C. Bie Absence of Develcpiental and Hierarciiical Cognitive Sciiesa 

Readers my wonder v*iy the instruiient fails to irKX>rporate 
criteria that r^resent a progressive hierarchy from lower to hi^ier 
level thoi^t, siKih as a Piagetian schenve from concrete to fontal 
operations or a Bloomian taxoncxry of cognitive objectives frm 
knowledge to evaluation. In addressing this issue Schrag (1987) has 
shown several reasons \ihy such schemes are unlikely to be helpful. In 
spite of extensive research, writing and attenptc to apply these 
schemes to practice, at least two serious problems remin. 

First, the wording of a task alone is insufficient for determining 
the extent to \Mch the student has to interpret, manipulate or 
evaluate information in new or i^n-algorithmic ways. Consider the test 
question, "Which of the follcwing would be the likely result of 
maintaining a hi^ protective tariff? (a) hi^er prices for domestic 
goods; (b) lower prices for foreign goods; (c) irxaneased foreign trade; 
(d) hi^i^ prices for farm products sold in, foreign markets." if the 
results of prat^±ive tariffs were described clearly in the text or 
class, this task would involve only recall, but if only the definition 
of a hi^ protective tariff were given, with no explanation of the 
historical and economic reasoning behind it, it would require students 
to make original inferences, ©lat is, the cognitive demancJs of a task 
cannot be estimated without knowledge of the tools and resources, or 
prior ej^jerience, the student has available to meet it. 

Secx)nd, in spite of advances in cognitive science and the study of 
human information processing, there is no consensus on the v;ays of 
categorizing cognitive operations and the extent to \*iich soae 
operations are inherently more cortplex and difficult than others, it 
has been shown that the apparently siitple act of decoding the meaning 
of a word requires many cognitive operations, Sternberg's (1983) 
analysis of the solving of multiple-<iioice analogy problems revealed 
seven main ccsnpronents: encoding, inference, mapping, application, 
ccsrparison, justification, and response, with this level of cooanplexily 
in trying to describe how the mind worlcs, it becomes ever more 
difficult to ooiipare the levels of thinking required by different 
problems or subject domains. Would a sofiiisticated analysis of the 
political effects of increasing tariffs necessarily involve more or 
less ccanfplex thinking than analysis of economic consequences? 

In short, the specific kinds of cognitive work generated by any 
given task or question depends largely i^xDn the often imknown resources 
and knwledge of the learner, and even if the nature of this work could 
be (tescribed with greater precision/ it is particularly difficult to 



were not systematically observed, but no striking differences on 
such matters were reported by researdiers. 
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jiistify the labeling of scane cognitive operations as more ccaiplex or at 
a higher level than others.2 

Althoai^ fine discriminations in levels of cognitive operations 
are difficult to make, we can still determine in a irore general vay 
vdiether students are being challenged to vise knowledge to go beyond the 
information given to solve non-routine problems. That is, vflien we knew 
what knowledge students are generally familiar with, \ghat materials 
they have at their disposal, vtet kinds of responses are ea^jecbed fitxan 
teachers, and how students behave in response to classroom teisks anJ 
questions, cur scales enable us to estiitate the extent to vMoh 
particular problems are intellectually challenging and the detent to 
vMch students and teachers approach the work in a thow^tful manner. 

D* Rating Scales vs Categorical ErequmcieB 

Schrag (1987) reviewed the advantages axid disadvantages of using 
rating scales and category schemes to record classroom discourse. We 
chose rating scales for three main reasons. 

First, and most litportant , categorizing and counting hi^y 
specific behaviors can yield misleading indicators of the quality of 
discourse; rating scales allcw more sensitivity to the context in \Aiich 
behavior occurs. For exaitple, one teacher mi^t ask 20 questions that 
would be considered "challenging" in the sense of requiring students to 
go b^ond the information given. But this teacher mi^t give students 
most of the answers after only si:5)erficial discussion. In contrast, 
another teaciher mi^t pose only one challenging question at the 
beginnir^ of class, but spend the entire class period prodding students 
to develop increasingly soffMsticated responses. Ihe second classroom, 
vMch had a ittuch lower frequency of challer^ing questions would be more 
substantively challenging than the first. 

To further illustrate the problem, consider scale #5, "Ihe teacher 
carefully considered reasons and conclusions." In making estimates 
about the nature and extent of this behavior, and ether forms of 
teacher-student interaction, it is often necessary to take into 
account three considerations simultaneously, each of vMch could 
technically be counted: the amount of time spent on the behavior; the 
number of discrete occurrences of the behavior; and the number of 
students involved. Such a recording scheme would be operationally 
cuntoer^ome (if not iitpossible) , and as indicated above the quantitative 
sum of these three dimensions could still give a misleading indication 
of quality. 

Second, some of the dimensions of interest are conceptually 
grounded in qualitative criteria that are liJcely to be violated by 



^ Further si^^port for our concerns with the limitations of 
cognitive developmental schema is provided by Keating (1988, in 
press) and Keating and McLean (1987) . 
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translation into a frequency count; for exairple, "Bie lesson displayed 
substantive coherence and cxsntinuity, " "ihe teacher was a model of 
thou^tfulness." For items such as these, it is difficult to imagine 
what mi^t be counted that mj.^t still capture the intent of the 
criteria. 

Finally, as indicated vmder guidelines, we wanted to generate 
criteria that mi^t eventually be used by teachers to reflect on their 
own teaching. Categorical frequency systems are likely to beccoe too 
ccnplex and elaborate for teachers to use, but a set of scales can be 
adapted to a more practical observation scheme. 

Using the scales to guide assessment of thcw^tfUlitess in 
dassrocras, we studied the teaching of social studies in five hirfi 
schools. 

lU Nsthodology 

■Ihe selection of departments and classes to be observed was 
described in the introduction. In each of the schools, three teachers 
were observed over nine classes, and six teachers once. Ihe classes 
were selected to inclxade students of diverse levels of school 
achievement. 

A. Reliability 

Eirawing fron a team of four researchers, two members gathered data 
at eacQi sciiool, and the two-person teams varied across schools. Durim 
eacQi visit they jointly observed one class from each of the three main 
teachers. Ratings were made indepenaently on each of the scales. 
Followii^ the lesson, discrepancies between ratings were discussed. lb 
estimate inter-rater reliability, Pearsonian correlations were csoraputed 
between the original ratings of 17 variables by each researcher on each 
lesson. For each visit the correlation between 51 ratings (3 lessons x 
17 s< les) was also ccstpited. As shown in Table 1, inter-rater 
agreement was well above .80 for most comparisons, and iinprovements 
occurred especially between visits 1 and 2. Ihe degree of rater 
agreement for each scale across all visits is indicated in Table 2 (see 
Table 3 for scale names and content). For fifteen of the seventeen 
scales _ raters agreed precisely more than 50% of the time. if the 
criterion is esqanded to include a difference of no more than one 
point, we find that all of the seventeen scales attracted rater 
agreement at least 90% of the time. 

B. SeLectiati of Minimal Criteria for Classroom Ohoug^itfUlness 

The e>?)loratory nature of the stuc^ led to creation of a large 
number of indices of thou^tfulness, but it would be useful to reduce 
these to a smaller number of essential variables. First, if the long 
list can be summarized into a more parsimonious set, eventually it will 
be more feasible for teachers to use jji reflecting \jpon and guiding 
their teaching. Second, a smaller set can be helpful in examining the 
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validity issue. Ihat is, if the scores of lessons on the minimal 
criteria also correlate with other criteria of thou^tfulness, we can 
have more conficJence that we are assessiixf an inoportant underlying 
feature of classrocaa discourse. 

Presumably, each of the scales represents a desirable 
cheoracteristic that would contribute to thou^tful discourse. 
Nevertheless, it could be x:iseful to make, a distinction between a 
criterion that indicates or helps to pronote hi^er order thinking 
versus one that, in addition, seems so essential that one could not 
imagine judging a lesson "thoughtful" unless the criterion were met. 
.Since we were not able to find analytic or eitpirical literature that 
conclusively determined a few key criteria, it was necessary to rely 
largely on deliberations within the research team. 

In deliberating on hew to select the most essential criteria, we 
put eacii scale to the following test: Based on the ccaio^ion of 
hitler order thinking outlined earlier, could a lesson conceivably 
score lew on this scale, yet still be considered a hic^y thou^tfta 
lesson? If the answer was "yes," then the scale was not considered 
critical as a minimal criterion. If the answer was "no," the scale was 
judged as beirg minimally necessary, thou^ perhaj^ not a sufficient, 
criterion for thou^tfulness. 

Without eiqplaining our decision for each scale, a few exanples 
will illustrate the reasoning used to disqualify four of the scales as 
minimal criteria. Teachers may conduct challenging large-group 
discussions in vdiicih they ask provocative questions, orchestrate 
student responses to each other, and offer iirportant information, but 
they may rarely interrogate individual students throug^i Socratic 
questioning that involves several teacher-student exchanges (#7) . A 
lesson ml^t carefully consider the reliability of primary sources in 
describing events that triggered the American Revolution but not 
necessarily result in students* offering original and unconventional 
soltiticais to the problem (#13) . A thou^tful discussion of the 
conflict between freedom of speech ani national securi-ly mi^t focus on 
the quality of reasoning found in two editorials without the teacher's 
integrating students* personal experience into the lesson (#4) and with 
relatively few students participating (#15) . 

On the other hand, it would be iitpossible to give recognition for 
thou^tfulness if the lesson failai to illustrate our defining feature 
of hi^er order thinking, that is, tasks or questions that pose 
cognitive challenge (#6) to go beyond the information given. 
Similarly, it would be inajpropriate to give a hi^ rating to a lesson 
that lacked substantive coherence about the subject of study (#2) , or 
one in vdiich students gave no reasons or e)5)lanations for their views 
(#12) . As a result of these kinds of considerations, we selected the 
following six minimal criteria: 

Sustained examination of a few topics rather than si:5)erficial 
coverage of many. (1) 
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Substantive cxjherence and continuity, (2) 

Students i^jere given an apprt:5)riate aiiomt of time to think and to 
respond. (3) 

The teacher asked challenging questions or structured ciiallenging 
tasks* (6) 

Ihe teacher was a model of thou^tfUlness, (10) 

Students offered ejqplanations and reasons for their conclusions. 
(12) 

Oliese criteria eventually formed the basis for assecising the 
degree of thouj^tfulness demonstrated in each of the 160 lessons, A 
case mi^t be wade for including other criteria; for exaitple, teacher 
careful consideration of reasons (#5), or students as qu^tioners and 
critics (#11) . Our intent here is not to settle the question of 
essential criteria for dassrocM thou^tfiilness, but only to develop 
one reasonable, manageable set that allows x:is to e5<plore the degree of 
variation among closes and schools, along with possible ©Q)lanatiQns 
for that variation.^ 

Ratings on the minimal criteria permitted us to identify the rcost 
and least thou^tful lessons as well as the teachers and the 
d^)artments vAiose lessons were rated consistently hi^ or low. Havii^ 
identified the essential criteria, the strat^y was to ^arch for other 
characteristics of lessons, teachers and schools that mi^t be 
associated with these indicators and that mi^t, therefore, sugge^ 
"causes" or interventions that would help teachers aid departments move 
from Icwer to hi^ier levels. 

17 Findings 

A. FreqtKSTcies and Relationships Azoong Dimensions of Thoug^xtfOlness 

Ihe schools were selected for their eitfiiasis on hi^ier order 
thinking, but the selection process did not allow for data collection 
to determine levels of thou^tfulness according the scales. We were 
first interested, therefore, in the frequencies of ratings on all the 
scales across all lessons. This information is given in Table 3. Some 
dimensions occur very rarely, especially Socratic questioning (#7), 
integrating student es^jeriences into the lesson (#4) , teachers trylr^ 
to get students to generate original ideas (#8), teachers questioning 
authoritative sources (#9), and students engaging in ttiou^tful 



Later it will bo shown that many of the scales correlate 
highly with one another. It is likely, therefore, that certain 
modifications in the set of minimal criteria would not 
significantly alter findings based the cariginal set. 
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disccurse vdth other students (#16)* Evidently even these select 
social studies di^rtinents iieglecl: these aspects of thoi^tfulnesse 

Cn the other haxxi, we famd a relatively hi^ occurrence of each 
of the six minijiial criteria and other diitensions as well; for exaitple 
students assuming the role of que^oner and critic, student 
contributions being articulate and germane, stuctent verbal 
participatJ.cati and involvement. If these d^)artraents are truly 
exenplary, hi^r frequencies on these characteristics would presumably 
disting'oish them from a more representative sample of schools, and 
this will be tested in future research. 

To \Aiat extent are the aspects of thou^tfulr^ss related? If a 
lesson shews a hi^ rating on cognitive challenge, is it likely also to 
show a hi^ rating on other dimensions? Table 4 presents correlations 
among the 17 scales . Generally, we find a hi^ degree of 
interrelationship among those scales that show variability across the 
five (or four) scale points. In contrast, the hic^y skewed dimensions 
of encw^ time to respond (#3), integration of student e35)eriences 
(#4), teacher awareriass of controversy in authoritative sources (#9), 
and students engaged in discourse with one another (#16) show little 
relationship to the other scales, if these low variance items were 
removed trm the correlation matrix, 62 of the 78 remaining 
correlations (80%) would be significant (p .05). Ihis si55)orts the 
possibility tiiat the 13 remaining scales may be ineasuring a single 
underlyii^ variable. later, in presenting findings on the differences 
between lessons that score hi^ versus low on the minimal criteria, it 
will be shewn t±iat these two groi?>s of lessons also vary in the 
ejqpected direction on several of ttie other criteria as well, vAiich 
lends fu2±her si?:port to the existence of an underlying quality. 

If these variables do represent an underlying construct of 
thoughtfulness, it is also possible that thou^±fulness itself may be 
ccanposed of sli^tly distinct diinensions. We e5?)lored this briefly 
throu^ factor analysis of the 17 scales. Ihis suggested two main 
coTOonents, one grounded primarily in teacher behavior'and the other in 
student behavior. Table 5 gives the results of a 3-factor model in 
which 8 of the variables loaded hi^y on a student participation 
factor (#1 which e}?)lained 24% of the variance) , and 6 of the variables 
loaded hii^y on a teacher thou^tfulness factor (#2 viiich esqplained 
23% of the variance) . Ihe third factor was not interpretable. The 
main factors mi^t represent two different teaching s^les: one in 
x*dch the teacher takes a hi^y directive role In guiding rigorous 
consideration of challengirg topics, and another characterized largely 
by idea-swapping or sharing among students with the teacher 
orchestrating, but giving less substantive direction to the discussion. 
Further analysis of relationships among the scales will be addressed 
in the future. 
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B* /lOXRBiting for Differences Between Kore and r^ci ahougfhtful 

Olia ultimate purpose of cxDnstructing quantitative indicators of 
classroaa thtxj^tfulness is to learn more about how to prcanote hi^er 
order thisMng in the classroom. To pursue this we examined the 
potential influence of other variables that may affect the degree of 
thou^tfulr^ss as nseasured by the six nunimal criteria. We estimated 
the relationship between levels of minlnal thou^tfulness and two 
general sets of variables: classrocm b^vior and practices that 
teachers can alter; and institutional and background variables such as 
the jjifluence of schools vs individual teachers, and students' age, 
school achievement: level, and r^ce-^thnicity. 

Ihe first step in this proqsss vas to create two grocps of lessons 
that r^resent, respectively, hi^ versus low levels of hi^er order 
thinking* To do this, we averaged each lesson's ratings on the six 
minimal criteria (HOfTAV) and then selected for conparison those in the 
top and bottcm quintiles.^ Based on this procedure, the hi^ groap 
included 36 lessons with a mean HOUftV of 4.60 (sd .21) and a low graap 
of 42 lessons with a mean HOOAV of 2.78 (sd .23) • The difference 
between the two groi:qps is substantial, representing 2*68 standard 
deviations of the ful]. groi?) of 160 lessons. Althou^ the lessons were 
drawn frcm social studies departments that all en^diasized hi'^ier order 
thinking, we see here considerable variation in the degree of classroom 
thoui^tfUlness amorg lessons. 

1. Tm lessons. 

To give a sense of the variation in actual classes, we describe below a 
lesson from each groip. 



. ^ We anticipated the possibility that average scores on the 
minimal criteria mi^t lead to inappropriate selections (e.g. a 
lesson with a 1 on teacher challenge, and a 2 coherence, but with 
5*s on the fair other variables shouldn't be considered hi^). 
Initially, therefore, we required specific levels for each 
criterion. "Hi^" lessons were those that scored 5 on tiiae and 
greater than or equal to 4 on few tcpics, substantive odherenoe, 
teacher challenge, teacher model, student explanations. "Low** 
lessons were initially defined as those that scored less than or 
equal to 4 on time, and less than or equal to 3 on the five other 
criteria. Ihis procedure resulted in selection of 18 (11% of the 
total) hi^ lessons and 12 (8% of the total) low ones, but several 
lessons in each group were tau^t by just a few teachers. Since we 
wanted to analyze a larger proportion of the lessons and include a 
greater variety of teachers, we cihanged the selection criteria to 
the HOIAV method. In coarparing these two selection procedures, we 
foRund that the averaging method did not result in inappropriate 
selections originally anticipated, and the pattern of differences 
betx^^een the hi^ and lew groi^js other variables remained the same. 
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Lesson 1 

m this US History class, the main topic was the nullification 
oontroversy between South Carolina and iTesident Jackson vAien the 
state declared ths tariffs of 1828 and 1832 to be null and void. 
Students had read textbook material on the backgrcwnd, but also 
excerpts frcm three source documents: South Carolina's "Adaress to the 
People of the United States," November, 1832 (anonymously written by 
Vice President John C. Calhoun); Andrew Jackson's "Nullification 
Proclamation," Dscsinber; 1932; and Calhoun's speech to the Senate, 
February 15-16, 1833, in defense of nullification and in opposition to 
the Force Bill then being considered in Congress. 

The teacher began by asking for students' reactions to the sourro 
readings. "Why are they more difficult?" she asked. Students had 
trouble articulating reasons, and she es^jlained that sources may seem 
more difficult than textbook reading, because, lailike textbooks, the 
political speeches were intended not to create an understaivJable story 
for 20th century students, but to speak directly to the people and 
issues of former times. 

The teacher then asked for definitions of key terms. 
"Nullification," and "tariff" were quickly defined, but it took a lorg 
series of patient questions to develop an understandirg of v4xy the 
tariff vculd benefit northern manufacturing states to the disadvantage 
of southern agricultural ones. During the discussion, students 
frequently spoke vtp vto they failed to understand, and the teacher 
tried to clarify the economic relationships by usii^ the contenporary 
analogy of the effects m various int^nssts of an iirport tax on 
Japanese cars. 

Sensing that the iirpact of the tariff was reasonably well 
understood, the teadier then asked students to examine the "Address to 
People of the iMted states" and to pick out the main arguments for 
nullification. One student summarized the main point that since the 
states had the power to form the Oonstitution f they also enjoyed the 
authority to nullify any actions of the Federal government not 
specifically delegated to it. Another student noted the point that the 
states' ratification of the Constitution included a special obligation 
to protect it fron usurpation of power by the Federal govemraent. The 
teacher pressed students to find further arguments, and they did: the 
claim that the Federal government may tax only to raise revenue, but 
not to protect scroe internal interests to the detriment of others;, and 
the argument that Jadkson's threat to use force on this issue would 
supercede, the law. In each instance, the teacher focused on wording in 
the text and asked students to give their a-.-n unierstarding of the 
argument. 

In a similar way students searched for Jacikson's arguments: 
nullification itself would be unconstitutional; that the states, by 
joining the Uhion, had given vp the ri^t to secede; that his election 
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ty^ cill the people, gave the Federal government powers beycrd the 
original power of states to enforce F-^Jeral actions. 

VJhen the alternative arguments seemed clearly ttnaerstood, the 
teacher asked students vdiether they could think of any conceivable 
situations in vMch they felt a state mi^t be justified in nullifyii^ 
a Federal action. Ihey had scsne difficulty in proposing specific, 
credible policies, tut th^ did suggest such situations as the Ftederal 
government vranting to tuild a nuclear base in a state; or Congress 
inaldiig smoking illegal (vMch would be opposed by tobacco producing 
state). Discussion of hypothetical situations failed to prxxJuce a 
conclusion on when nullification mi^t be justified, but the teacher 
alerted students that the issue will cxsive again. 

Ihis lesscai received hic^ ratings for focusing on a few topics, 
substantive coherence, enou^ time to respond, careful consideration of 
reasons and eaqjlanations, Socratic questionii^, teacher nodeling 
thou^tfulness, students' giving esqjlanations and reasons, students 
generating original ideas, student coanments beii^ articulate and 
germane. More than 75% of the students participated, and bet^veen 50% 
and 75% showed genuine involvement. Ihe HOTAV score for the lesson was 
4.5. 

lesson 2 

An Advanced Placement US History class considered the South 's 
resistance to the abolition of slavery. The teacher began by outlining 
vAiy three approaches to the abolition of slavery each seemed unfeasible 
prior _ to_ the^ civil war: sending slaves back to Africa; gradual 
emancipation with financial coaipensation to slave owners; and immediate 
legal abolition. None of these were discussed in much detail, but 
after the teacher indicated that in the long-run, slavery was not 
econcanically feasible for the South, students began askirg a number of 
questions: Htow much did slaves cost? V3hy didn't the South rely on 
cheaper immigrant labor vMch would also give them more representatives 
in QDngre?''^ What was happening in the cotton business? Didn't the 
South s^ chey were being left behind (in econondc growth) by the 
North? 

The teacher responded to the questions with a set cf short 
lecture on the Southern cultural attachment to the institution of 
slavery, the relationship of British trade to the economy, and the 
resources (human, capital, technical) that would be needed for the 
coming industrial revolution. 

After recognizing these as sli^t digressions, the teacher 
returned to describe the abolitionists ard their 'movement. She 
described their writing, and eitfjhasized how they used the media to 
influence public opinion. vflien she pointed out that even the 
abolitionists did not siqpport full equality for blacks, a student asked 
viien fuller equality was achieved. In another short lecture, the 
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teacher reviewai the long history of gaining legal ri^ts for blacks 
thrco^ the 1960's, 

A stident renarkad that it took the US at least 100 years to 
develop equal ri^ts in the law, and that it seens that scoe people 
e^qpect South Africa to do this ovemi^t, Ciis lad to teadier and 
student conments on parallels and differences between the US and South 
Africa, Students also referred to British resistunoe to colonial 
independence (especially Ir^ia) • A wide-rangii^, unfocused discussion 
ensued on iiiperialism and foreign policy issues related to Ireland, 
Israel, Iran, and Nicaragua. Althcxjc^ the ccnroon the:oB seeraed to be 
inportance of eooncrdc self interest versus scroe hi^ier morality, the 
discussion was not directed to examine this systematically. 

In this lesson, students consistently asked provocative questions, 
they generated original ideas, and they were articulate. More than 75% 
participated verbally, hit less than half showed genuine involvement, 
ax)d usually students failed to e>q)lain or give reascais for their ideas. 
Bie teach^, vMle responding in an academically respectable way to 
student inquiries, did not pursue any topic in depth, nor did she pose 
challenging tasks for the student. She did not press for careful 
consideration of her own or the students' reasons, and she indicated 
more of a sense that she had the answers than a spirit of 
reflectiveness in working throu^ a problem. Ihe IX7IAV score for this 
lesson was 2.5. 

2. General Differences between Hi^ and low Lessons. 

Mcrvlng from these exaitples, to the mean scores for the hi^ and 
lew lessons, we consider new how the hi^ and Iw lessons differ on 
variables other than the minimal criteria for thoi^tfulness. Table 6 
shows cxsaparisons between the hi^ and lew grocqps on each of the 
scales. 5 To gain a sense of the relative size of differences between 



^ The entire pool of lessons included nine lesson 
observations fran 15 teachers and one lesson observation from 30 
teachers. To 'maximize the number of cases, we used the entire pool 
of 160 lessons for quantitative analyses. To determine vAiether 
results waild be biased by overr^resentation of lessons frm the 
15 primary teachers, we conducted parallel analyses using only one 
set of lesson ratings from each of the 45 teachers* Far this 
purpose, prJinary teachers" ratli^ across their nine lessons were 
averaged to create a single lessen profile. Diese "unwei^ted" 
analyses yielded findii^ consistent with those In Tables 6, 9, 12, 
and 13, and we concluded that vise of the entire pool of lessons 
produced acc^jtable estimates. 
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groups, the table e^qoresses the differences between xasans in terms of 
the standard deviation of all lessons.^ 

Of the 11 non-ioiniinal criteria, the hi^ and lew lessons seem to 
differ most on five dimensions: teacher careful consideration of 
student reasons (#5) , teacher Socratic questicmng (#7) , student" 
contributij>g original ideas (#13), students beirg articulate and 
germane (#14), and students being involved in the lesson (#17). For 
each of these dimensions the mean difference between the groijps 
exceeded 3, point on a five-point scale, and the effect size for each 
also exceeded one standard deviation. 

VBiile sane differeix:es appear on the other six scales, it would be 
difficult to make a case that any of them are substantial, for no 
others reach a one standard deviation degree of difference. A number 
of interpretations mi^t be made of these results, but we wish to 
hi^i^t three. First, only two criteria with clear inplications for 
tea<3ier behavior stand out: teacher careful consideration of the 
qua].ity of student reasons, and Socratic questioning. We inclvded in 
the scheme three other te^cher behaviors that mi^t theoretically mi^t 
appi^ar to prcncte 15:ou^tfulnGss integrating student experiences into 
the lesson, enccuragii>g students to contribute original ideas, ana 
shewing awareriess that not all authoritative sources provide adequate 
answers. None of these, however, distinguished between the hi^ an3 
lew lesson grojps. 



^ Results for analysis of variance are given in Table 6 for 
the reader's information, bitt not discussed, because it is 
problematic as to vdiether the set of lessons can be considered a 
random sanple of a larger population. In a technical sense, 
lessons were not randomly sairpled, either from the full universe of 
hi^ schcx>l social studies lessons, or from all the lessons in the 
select d^)artments. On the other hand, one mi^t assume that the 
select d^-^Artments do r^resent a larger universe of departments 
enphasizing hi^er order thirilc'z^g. Furthermore, since the observed 
lessois involved a diverse ra^ige of students and courE'":^, occurred 
at several points in time, and were selected primarily acconJing to 
sdtieduling convenience, rather than teaching technique or topic, 
they mi^t be assumed to represent the larger universe of lessons 
in the departments. In any case, with this small saijple we believe 
the size of the differences is a more meaningful indicator than ti*e 
level of statistical significance. 

Bie cotparisons between the hi<^ and low grojps are 
corroborated by the Pearsonian correlatiais of the other scctles 
with HOOAV as shewn in Table 4. Eaportion of students \^o tcdk 
(#15, r=.34) appears a little stronger here, but this may be 
largely a result of its own correlation with articulateness and 
germaneness which is .52. 
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Second, neither the cuoint of student questioning and cariticism 
(#11) , nor the proportion of students v4io pecticipate (#15) , differed 
substantially between the hi^ and low groi^js. Ihis supports the point 
that the quality of student participation is probably a better 
indication of classrocm thou^tfulness than the quantily. Biis is 
further reinforced by the fact that indicators more li3cely to tap 
qualily, namely, student originality and student articulateness and 
germaneness, did differentiate in the expected direction. ^ 

Biird, given a ccnimon observation that many students may resist 
hitler order thinking because they are presumed to prefer more 
structured routines of school that may seem easier to master, it is 
encouraging to find here that in the more thoughtful lessons a higher 
proportion of students consistently shewed signs of engagement (#17) 
with the material. 

Did the hi^ lessons differ systeneitically from the low in certain 
generic teaching practices such as the use of lecture, the type of 
reading assigned, or the kinds of writing done? Each lesson was 
categorized according to the presei'jce or absence of certain practices. 
We asked vdiicii of the follcsdng was the doninant practice during the 
lesson: lecture, film, recitation> tescher-cettered discussion, 
student centered discussion, student grou^jwork, individual student 
seatwork, oral r^rts, and other. Types of reading were classified as 
tesctbooiks, articles, primary sources, literature, other and none. 
Writing done in class was categorized accordli\g to outlining, notes, 
worksheets (ccoplete sentences required), ccaatplete sentence answers, 
paragraph or essay, research paper, other. Eadi of the reading and 
writing items received a score of i if it occurred during the lesson, 0 
if it did not. 

Table 7 shews the results of ccarparison between the hi^ ana low 
grcn^js. Differences appeared in only a few areas. The dominant 
practice in the more thco^tful lessons was teaciier-oentered 
discussion, but the less thou^tfUl lessons were dominated by lecture, 
recitation, and they" "also included teacher-centered discussion. More 
thoucfitful lessons relied less on textbooks and more ai primary sources 
and other types of reading. The less thou^tfUl lessons relied largely 
on a textbook. 

Ihe findings confirm v*iat mi^t be expected: In contrast to 
lasture and recitation, teacher-centered discussiais are more likely to 
challenge students to go beyond the information given, as is the 
reading of primary sources, in contrast to textbook reading. One mi^t 
e)5>ect the more thoucjitful classes to involve more extensive prose 

2 2he prt^rtion of tiite students were engaged in thoughtful 
discussion with one another (#16) mi^t be considered a oenibination 
of quantity and quality. Its failure differentiate between the 
qcojgs can probably be attrlbutt.! to its extremely rare occurrence 
(it had the Icwest mean of all student behavior itene) . 
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writing, but in neither groap of classes did we observe rnich writing 
other than note-taking and co9:tpletion of short answer exercises. 

Ihere were no substantial differences in the forms of writing dot^ 
between the Hore and less thou^tful lessons, but we found scane 
differences In the quality of written tasks. Wtirten assignraents 
distributed during our observations were collected, and the tasks were 
coded according to the extent to \Aiich they required students to (a) 
draw inferences, (b) give reasons, (c) integrate information frm a 
number of sources, (d) develop an idea or theme; also, \*ether they 
allowed students to generate original responses^ and vfliether students 
had an appropriate amount of time to costplete the task. Ratings on 
each task were added to produce a hi^ier order challenge score. 
Althco^ the scores of all tasks were rather low (less than 2 on a ap- 
point scale) , tasks assigned in the top quintile of lessons did exceed 
those in the bottom quintile by about .75 of bottom groip's standard 
deviation. 

3. HbB iDfhxanca of institutioncil and background factoirs en classrocBi 
tlxxi^htfulness. 

Having found soooae aspects of teacher and student behavior 
associated with differences between the hi^ and low classes, it is 
important to ask about the influence of institutional and bacikground 
factors. Is it possible, for example, that even with considerable 
variation among teachers, scsae social studies d^)artra9nts have 
consistently more thou^tful lessons than others? Would teachers 
assume that older students are more capable of hi^ier order thinking, 
and therefore, enjliasize hi^er order thinking more frequently in 12th 
graJe than 9th grade classes? Wcwld classes with a preponderance of 
hi^ adiieving students reflect consistently more thou^tfulness than 
classes with large proportions of low achievers? Wbuld the proportion 
of minority students in a class tend to decrease the level of 
thou^tfulness? Are elective courses more likely to proaaote 
thoo^tfulness than required courses? We examir^ these issues by 
regressing BOTAV on the relevant variables. 

a. IbB lopact of Sdiool. 

Hie school means and standard deviations on HOGCAV given in Table 8 
show that the five schools do differ. Indeed, they can reasonably be 
seen as two groiqps, with Carlsberg, Grandville, and Arnold 
distinguishijig themselves from Bradley and Scarborou^. Regression 
analysis (Table 9) indicated that differences between schools accounted 
for about 26% (adjusted r square) of the total variance among all 
lessons. While most of the variance in individual lessons is due to 
other factors, even in this grcfop of presumably exenplary departments, 
a significant portion of the variance can be attributed to d^)artmental 
(or school) differences. Ihe paper by IfcCarthy and Schrag e55>lores how 
these differences mi^t be explained by different patterns of 
institutional si?)port. 
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b. TitpfvA of Oia class grade level; stisients' anrsrage school 
aciiieweBerit level, peiroent miixarity, and reqairei or elective status. 

For each lesson, teachers indicated the percentage of stuaents in 
enrolled in the 9th, 10th, 11th, and 12th grades; the percentage of 
students v*ose grade point average placed thaa in the lowest, nddaie 
and hi^Mst one-third of the school achievement distribution; the 
percent of minori-ly students (black, hispanic, asian, other); and 
vifliether the class was required or elective. Ihe Pearsonian 
correlations between HOflAV and each of these variables is given in 
Table 10 . 

The proportions of students in the different grade levels are 
inconsistently associated with classrocm thou^tftOness. Althou^ 
there nay appear to be an advantage in having lots of 10th graders and 
few 11th graders, these relationships are not robust. Ohe prcportion 
of students in the lowest, middle, and highest thirds of school 
achievement has no relationship to hi^ier order thiiUdng, vMch helps 
to establish the prospect for prcooting thou^tfulness equitably for 
students of all achievement levels. Ihe results on minority 
ccnposition of the class indicates less higher order thiridng with 
larger proportions of blaciJcs, and more with larger proportions of 
hispanics, but these results should be interpreted in the li^t of the 
very small proportions of all minorities other than blaciks and their 
overall distributions in the schools as shown in Table 1 (Introduction) 
and in the observed lessons as shewn in Table 11. In the observed 
lessons, the T tHx imm percent for hi^)anics, for exaaple was 15% or 4 
students in a class of 25. 

To estimate the inpact of these background factors independent of 
the variance th^ share with one another, they were regressed together. 
To sinpli^ the analysis, ccstposites were created for grade level and 
achievement level, and the mirority cca*3entration variable was 
restricted to blaciks. Table 12 indicates that together, these 
variables account for only a small amount of the variance in cleissrocm 
thcu^itfUlness (adjusted r square = .11) , and that this is due largely 
to the sli^t negative relationship associated with the prcportion of 
black students in a class. Because of -ttie small magnitude of this 
finding, we should not make too much of it, but it does suggest that 
independently of students' achievement level, teachers ej^x^ classes 
with large proporticais of blaciks to less higher order thinking.^ 

Exc^ for this finding, the results are most enoouragii^, because 
they illustrate that three other cosnmonly paxieived obstacles to 
prctnotii^ higher order thinking (student ycutii, low school achievement, 
required courses) need not stand in the way. lhat is, in these 



y On average, an increase of one standard deviation, or 24%, 
in the percentage of blacks in a class could be expected to 
decrease the lesson's HOTAV by .38 standard deviations, or by .26 
points on the five point scale. 
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schools, chosen for their entphasis on hi^ier orcJer thinking with a 
diverse population of students, the level of classroon thou^tfulness 
is due to other factors, primarily, we assuite, to the ccmimitnient and 
caraft of individual teachers. 

Althcw^ the background variables as a groap have no inpact on 
lesson thaaj^tfulness, it is possible that their infliience aid that of 
the schools mic^t change v*ien the institutional and bacikground 
variables are considered in one regression locxJel. For exanple, if scsre 
background variables such as proportion of minorities in classes or 
proportian of ninth graders in classes are much hi^:^ in scrae schools 
than others, one would want to examine their effect on thou^tfulness 
independent of the iirpact of the school culture. Table 13 presents 
this information. 

After taking the school into account and comparing this 
infoinnation witii Table 12, grade level and required course new shew a 
possible (thcu^ minuscule) influence on thoui^tftilness, favoring 
younger students and elective courses. In contrast, the influence of 
students' achievement level disappears even further, and the effect of 
prcportion of blacks is also reduced to non--significance. By 
considering the iirpact of school and background variables together, we 
see also that the differences between schools have been sli^tly 
reduced (ccctpare with Table 9) . We can also probably assume that the 
previously noted effect of proportion of blacks was probably due to 
that fact that the school scoring Icwest on thou^tfulness also tau^t 
classes with the hi^iest proportion of blacks. By addir^ background 
characteristics to the model, we have es^lained only an s<aaitional 1.5% 
of the variance in lessons' hi^er order thinking (subtracting the 
adjusted r square of Table 9 from that of Table 13). This fu2±her 
underscores the inpact of institutional (possible departmental) 
culture. 

V Oonclusion 

We began by reviewing a conception of hi^oer order thinking that 
enfiiasizes teadiing the knowledge, skills and dispositions necessary to 
meet non-raitine intellectual challenges. Next we proposed that in 
order to iorprove practice in this direction, it would be useful to 
assess levels of classroom thou^tfulness in ways that would 
distinguish betaken lessons that are more and less successful in the 
prtsnotion of thinking. 

Bie enpirical stucty focused initially, therefore, on the 
methodological question, "What dimensions of teacher and student 
discourse can be used to describe the degree of hi^er order thinking 
promoted in social studies classes?" We identified seventeen possible 
dimensions and found that they could be reliably coded. The overall 
ratings in 160 lessons showed that several of these dimensions rarely 
occurred, but roc^ of them manifested reasonable variance and 
correlated with one another in ways that would be expected from 
previous research and e3?)erience. Exploratory factor analysis 
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suggested the existence d£ two itain teaching styles: one in ^Mdi the 
teacher rigorously presses students to e)^lain their ideas and one in 
\diich the teacher guides the discussions less directly by encouraging 
more free-vSieeling student interaction. 

2he second main question was, ''What teaching practices, 
characteristics of schools and characteristics of students tend to 
distinguish between the most and least thou^tful lessons." Six 
dimensions were selected as minimal criteria for classrocm 
thcu^tftilness. Ocetparisons of the tcp and bottcai 20 percent of the 
lessons on these variables indicated that two main teacher behaviors 
(careful consideration of reasons and e35)lanatians; and Socratic 
questioning) , and two main student behaviors (generation of original 
ideas, and articulateness and germaneness) are most associated ^th 
scores on the six essential criteria. It was also encouraging to find 
that students were more engaged in the more thoughtful classes. Ihe 
most thcw^tful lessons were dominated by teacher-centered discussion, 
and the least thcu^tfUl more by lecture ard recitation. Ihe most 
thcu^tfUl lessons also involved less reliance vpon textbooks and more 
vpon primary sources and other forms of reading. 

Doe level of classroom discourse in lessons varied as a function 
of the school (department) in \*iich the lesson occurred, but did not 
vary according to the grade level of the lesscai, prcaportion of minority 
students, the general achievement level of students in the class, or 
whether the course was required or elective. Although, differences 
between sdiools had clear inpact, most of the variance in classroom 
thoughtfulness was due to unmeasured factors, and the most powerful of 
these probably relate to teachers* individual conntdtments, orientations 
and skills. Ihe paper by Onosko coirpares teachers vSiose lessons show 
most consistent eitgphasis on thinking with those vAio have less 
consistent success. 

Since the stud^ was confined to only five schools, chosen for 
their presumably exenoplary eomphasis on hi^er order thinking on a 
department--wide basis, these findings may have only limited 
generalizability. Future^ research will examine the applicability of 
the findings in two additional sets of schools: a r^resentative groap 
in which no school clp-'ms a depa2±mental-wide eicphasis, and a set of 
schools that claim to eirphasize hi^ier order thirfdi^, but that have 
also made significant changes in the organization of instruction in 
order to ftalfill this purpose. Examining levels of thou^tfulness, the 
influence of institutional and background factors, and differences 
between teachers in the three sets of schools should give us further 
dues as to hew to promote hi^ier order thinking in social studies. 
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Table 1 



Rater BeOiability 
(Baarscnian GoErelaticaB) * 



School 
Arnold 

Bradley 

Carlsbercj 

Grandville 

Scarfaorouc^i 



Visit 1 
■^^sscm Ptaoled 
1 .77 



2 .90 

3 .93 

1 .78 

2 .76 
3 

1 .84 

2 .87 

3 .70 

1 .94 

2 .91 

3 .96 

1 .51 

2 .57 

3 .68 



.88 



.79 



.81 



.94 



.74 



Visit 2 

Pooled 

.87 



.85 
.91 

.79 
.90 
.80 

.82 
.93 
.96 

.95 
.90 



.96 
.74 
.88 



.90 



.84 



.90 



.93 



.87 
.97 
.90 

.90 
.87 
.90 

.95 
.91 



Visit 3 
Pooled 

.91 



.86 



.70 
.83 
.87 



.90 



.92 



Mean of Pooled 
Across Visits 

.90 



.84 



.82 



.88 



.94 



.81 



Mean across lessons = .85 
Mean of pooled data = .87 



-^a?&^ SLd^S^i^LiLg^^SooTiSit^ ' 
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Table 2 

Inter-Rater Agreement for Each Scale* 



Scale 


% Exact Aareement 


% Differ by 


1 


59 


93 


2 


64 


100 


3 


46 


97 


4 


67 


100 


5 


54 


95 


6 


59 


92 


7 


54 


97 


8 


46 


90 


9 


85 


92 


10 


64 


97 


11 


56 


•31 


12 


64 


95 


13 


62 


.97 


14 


82 


100 


15 


87 


100 


16 


92 


100 


17 


64 


100 



* % agreement is based on 39 ratings per scale Jay two raters. 
Pairs of raters varied among 4 researchers. 
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Table 3 jj^26 
Nature of Discourse In 160 High School Social Studies Lessons (5 high schools) 

Findings are 



^In^t^TM^^^^"* 1 : ^* ' " "^^'^y Inaccurate" description of class; 5 - "very accurate, 
reported as the percent of lessons receiving each rating. accurate* 



*1. 

FEHTOP 

*2. 

SUBCOH 

*3. 
TIME 

4. 

STUEXP 

5. 

ICONS 

*6. 

TCHAL 

7. 

TSQC 

8. 

TORIG 

9. 

TAWAR 



*10. 

TMOD 



11. 

SCRIT 

*12. 

SEXPL 

13. 

SORIG 

14. 

SARI I 

15. 

TALK 

16. 

DISC 

17. 

INVO 



In this class, there was sustained examination of a few topics 
rather than a superficial coverage of many. 

In this class, the lesson displayed substantive coherence 
and continuity. 

In this class, students were given an appropriate amount of 
time to think, that is, to prepare responses to questions. 

In this class, students* personal experience (where relevant) 
was integrated into the lesson. 

In this class, the teacher carefully considered explanations 
and reasons for conclusions. 

In this class, the teacher asked challenging questions and/or 
structured challenging tasks (given the ability level and 
preparation of the students). ^ 

In this class, the teacher pressed individual students to 
justify or to clarify their assertions in a Socratic manner. 

In this class, the teacher tried to get students to generate 
original and unconventional ideas, explanations, or solutions 
to problems. 

In this class, the teacder showed an awareness that not all 
assertions emanating from authoritative sources are absolute 
or certain. 

In this classroom, the teacher was a niodel of thoughfulness. 
(Principal indications are: the teacher showed appreciation 
for students* ideas and appreciation for alternative approaches 
or answers if based on sound reasoning; the teacher explained 
how he (she) thought through a problem, the teacher acknowledged 
the difficulty of gaining a definitive understanding of the topic. 

In this class, students assumed the roles of questioner and critic. 

In this class, students offered explanations and reasons for their 
conclusions. 

In this class, students generated original and unconventional ideas, 
explanations, hypotheses or solutions to problems. 

In this class, student contributions were articulate, germane to 
the topic and connected to prior discussion. 

What proportion of students were active participants? 

What proportion of time did students spend engaged in thoughtful 
discourse with each other? 

What proportion of students showed genuine involvement in the 
topics discussed? (Cues include raising hands, attenti veness 
inanifested by facial expression and body-*language, interruptions 
motivated by involvement, length of student responses). 



1 
0 

69 
10 



47 
40 

64 



14 



17 



59 



2 


3 


4 


5 


8 


22 


29 


41 


23 


28 


28 


20 


2 


9 


44 


44 


21 


7 


3 


1 


32 


30 


15 


13 


12 


16 


32 


39 


28 


15 


6 


5 


38 


14 


6 


3 


13 


17 


3 


4 


21 


29 


28 


20 


24 


CD 


16 


C 1 


26 


20 


34 


15 


13 


48 


22 


1 


14 




*i 1 




0-25% 
4 


25-50X 
15 


50-75% 
40 


75-101): 
41 


1-25X 
28 


25-50% 
6 


50-75% 
5 


>75% 
3 


2SX 
14 


25-50% 
28 


50-75% 
29 


>75% 
29 



♦These variables are considered minimal requirements for a thoughtful lesson. 
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Table 4 



Correlations Among Classroom Observation Scales 
(£< .05, N = 160) 



1. 

2. 

3. 

4. 

5. 
' 6. 

7. 

8. 

9. 
10. 
11. 
12. 
13. 
14. 
15. 
16. 
17. 



FEWTOP 

SUBCOH 

TIME 

STUEXP 

TCONS 

TCHAL 

TSOC 

■TORIG 

TAWAR 

TMOD 

SCRJT 

SEXPL 

SORIG 

SARTI 

TALK 

DISC 

INVO 

HOTAV 



1.00 



.44 
1.00 



3 4 5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


TIME STUEXP TCONS 


TCHAL 


TSOC 


TORIG 




TMOD 

1 1 l\JU 


SCRIT 


Ou Ar L 






TAi y 


UloL 


T Mlf A 


ijnTA\/* 
HOTAV^ 


.42 


.47 


.28 


.30 




.42 




.45 


.31 


.27 


- 


- 


.30 


.73 


.62 


.28 


.44 




.24 


.64 


- 


- 


.28 


.24 


- 


- 


- 


.67 


1.00 - 


"* 






— 


- 


- 


- 


- 


.22 


- 


- 


- 


- 


1 .00 








"* 










- 


- 


- 


- 


- 


1.00 


.44 


.61 




■* 


.79 


— 


.43 


.25 


.37 


- 


- 


.22 


.71 




1.00 


.34 


.35 


*" 


.50 


.27 


.60 


.36 


.42 


.30 


.24 


.43 


.74 






1 .00 


"* 




.52 




.33 


.24 


.28 


.21 


- 


- 


.51 








1.00 


** 




.38 


.22 


.48 


.35 


.35 


.37 


.46 


.28 










1.00 


.21 






— 


— 


- 


-.22 


- 


- 












1.00 




.45 


.33 


.41 


- 




.32 


.79 














1.00 


.43 


.42 


.60 


.46 


.39 


.52 


















1.00 


.46 


.63 


.43 


.24 


.51 


.73 


















1.00 


.67 


.55 




.48 


.48 




















1.00 


.52 




.57 


.56 






















1.00 


.35 


.51 


.34 
























i.no 


.28 




























1.00 


.44 




























1.00 


subcoh, time, tchal, tmod, sexpl. 
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Table 5 

Principal Components Analysis of 17 Variables of Thoughtful ness* 

■ 

nOTATED LOADINGS 





1 


5 


3 




0.812 


-0.1^0 




SARTI 


0.766 


0.322 


O PIS 


IMVO 


0.7'»0 


0.219 


O.0'« J 


TALK 


0.725 


0. 107 


0.070 


SGRIG 


0.697 


0.291 


0.256 


SEX PL 


0.605 


O.^iBl 


0 . 00 1 


TORIG 


0.576 


0. 162 


-0.3'«3 


DISC 


0.530 


-0. 130 


-0.597 - 


ICONS 


0.031 


0.870 


0 . 1 33 


TMOD 


0.113 


0.8A8 


0 . 23'» 


SUBCOH 


-0.0B5 


0.777 


0. im 


TSOC 


0 . 052 


0.698 


0.032 


FEWTOP 


0 . 2,in 


0.63.2 


"0.272 


TCHAL 


O.'iL^'? 


0 . 59fi 


-0. J72 


TAWAR 


-O . 007 


0. l^Q 


0.659 


!5TUEXP 


0. 193 


-o.o;yi 


0 . ft (>< t 


TIME 


0 . 25/) 


-0 . 057 


■0.056 



VARIANCE EXPLAINED BY ROTATED COMPONENTS 



1 S 'i 

'♦.ISO 3.006 l.'lOl 

PERCENT OP TOTAL VARIANCE EXPLAINED 

1 2 3 

2^. '♦09 22.850 n.2'»l 

* 3 components were retained and rotated through VARIHAX 
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Table 6 



Ccmparlsons Bet»«en T<^ and Bottom Quintiles of 
Lesson Thougftitfulness (HOTAV) on All Scales 



Variable 



*1. 
*2. 
*3. 

4. 

5. 
*6. 

7. 

8. 
, 9. 
*10. 

11. 
*12. 

13. 

14. 

15. 

16. 

17. 



HOTAV 

FEWTOP 

SUBCOH 

TIME 

STUEXP 

Txxm 

TCHftL 
TSOC 
TORIG 
T/ff'2AR 

m>D 

SCRIT 

SEXPL 

S0RI6 

SARTI 

TALK 

DISC 

INVO 



^^^h (S.D.) ^ 



4.60 
4.38 
4.53 
4.56 
1.56 
4.19 

SO 
2.83 
1.94 
1.67 
4.67 
2.94 
4.22 
3.08 
4.33 
3.42 
1.58 
3.19 



.21) 
.38) 
.70) 
.56) 

1.08) 
.82) 
.«2; 

1.80) 
.80) 

1.17) 
.54) 

1.17) 
.59) 
.97) 
.59) 
.65) 
.81) 
.86) 



Low (N=37-i 



2.78 


{ .23) 


.68 


2.68 


2.92 


( .89) 


.99 


1.93 


2.46 


( .77) 


1.09 


1.90 


4.16 


( .83) 


.72 


.56 


1.35 


( .59) 


.81 


.26 


1.86 


( .71) 


1.18 


1.97 


2.60 


( .83) 


1.07 


2.04 


1.19 


( .40) 


1.14 


1.44 


1.27 


( .51) 


.99 


.68 


1.24 


( .50) 


1.09 


.39 


2.32 


( .58) 


1.07 


2.20 


2.70 


( 1.39) 


1.34 


.18 


2.19 


( .84) 


1.14 


1.78 


2.00 


( 1.00) 


1.01 


1.07 


3.08 


( 1.09) 


.95 


1.32 


2.70 


( 1.00) 


.84 


.86 


1.38 


( .54) 


.98 


.20 


2.05 


( .97) 


1.04 


1.09 



35.75 
11.86 
12.05 
2.36 
1.01 
12.94 
14.08 
8.00 
4.35 
2.02 
17.93 
.80 
11.89 
4.63 
6.09 
3.61 
1.28 
5.32 



* HOTAV is the sum of these items. 

** SDp is the standard deviation of all 160 lessons in the seimple. 
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Table 7 

Frequencies of Classroom Practices 
High and Low Lessons 



Variable 


Hleh (NsSfi") 




Pearson Chl-Sqiiaro 


DLECT 


1 


1 D 


19.94 


DFILM 


n 
0 


0 





DRECI 


n 
0 


14 


16.85 


DTCHD 


33 


1 A 
xo 


19.39 


DSTUD 


0 


x. 


.99 


DGRUP 


8 




4.62 




2 


3 


.19 


DREPT 


0 


0 




DOTHR 


0 


0 




RTEXT 


8 


20 


7.82 


RARTC 


2 


4 


.67 


RFRIM 


13 


5 


5.02 


RLITR 


0 


0 





ROTKR 

RNONE 

WOUTL 

WNOWN 

WWORK 

WCOMP 

WPARA 

WPAPR 

WOTHR 

WNONE 



12 
6 
0 

24 

11 
7 
1 

0 

1 

4 



6 
9 
0 
19 
12 
3 
3 
0 
0 
7 



2.88 
.66 

1.77 
.03 
1.98 
1.00 

1.04 
.87 



Probability 
.000 

.000 
.000 
.321 
.037 
.666 
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.005 
.414 
.025 

.090 
.418 

.184 
.863 
.159 
.317 

.307 
.351 



TiETTS 

School Differences in HOTAV 

SCHOOL 

Arnold Bradley Carlsberg Grandville Scarborough 

rfean SD Mean SD Hean SD Mean SD Mean SD 
HOTAV- 3.91 .60 3.50 .68 4.13 .48 3.99 .57 3.15 .58 
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Table 9 



Regression of ixmN cn Schools* 
(»=160) 

WI^JHER: .528 SCPKRED MDIinPIE R: .278 

ADTOSTED SCSIfiRED MOHEIEIE R: .260 STftNDM® EFRCS OF ESTIMftTE: 0.586 





SB) mm 


SID CO^ 


T 


P(2 HKH^ 


3.151 


0.105 


0.000 


29.910 


0.000 


0.764 


0.147 


0.455 


5.206 


0.000 


0.344 


0.148 


0.203 


''..329 


0.021 


0.980 


0.148 


0.577 


6.629 


0.000 


0.839 


0.148 


0.494 


5.677 


0.000 



VKREAffiJE 

OQNSISVNT 
AR& 



AK2VIYSIS OF VAKiaiKE 



I®3?ESSiaj 20.562 4 

RESHXiaL 53.313 155 



5.140 
0.344 



14.945 



P 

0.000 



^essOTs were assigned dumry values, 1 or 0, for each of the variables: ARA (Arnold) , BRK (Bradley) 
caR (Carlsberg), GRN (Grandville). Scarborou^ was the "constant." Coef f icients rtpresent the 
difference an the HCO^AV mean between each school and Scarborou^; for exanmle, Arnold seeds 
scarborougn's mean by .76. " 
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Table 10 

Correlations of HOTAV with 
Lesson Background Variables 

(N=160) 



% of Students 

9th - .08 

10th .24 

nth - .22 

12th .15 

low :/3 . .07 

mid 1/3 .07 

high 1/3 . .01 

black. . .36 

Hispanic .40 

Asian .18 
other minority . - .05 

course required (1) or 

elective (0) .02 
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Table 11 

Distributions of the Percent of Minority Students 
in 160 Lessons 



BLACK ASIAN HISPANIC OTHER 



Minimum 


0 


0 


0 


0 


Maximum 


81.0 


42.0 


15.0 


8.0 


Mean 


16.6 


10.0 


2.4 


0.3 


Standard Oaviation 


24.2 


10.9 


3.5 


1.3 
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Table 12 

Regressi<xi of HOTAV on Lesson Background Variables 
MOL TIPLE R: .370 S(yJARED MULTIPLE R: .137 

ADJUSTED SQUARED MULTIPLE R: , ;i4 STANDARD ERROR OP ESTIMATE: 0.641 



VRRZABLB OOESFICIBUS SID ERROR SID OCEF TQLBRANCE 



CC»1STANT 
GRADEAV 
ABILAV 
BLACK 
DIMREQ 



4.071 
0.002 
-0.085 
-0.011 
0.013 



0,574 
0.053 
0.091 
0.002 
0.123 



0.000 
0.003 
-0.072 
-0.381 
0.008 



1.0000000 
.9666467 
.9384370 
.9336964 
.9675763 



7.090 
0.043 
-0.932 
-4.930 
0.108 



p(2Tm.) 

0.000 
0.966 
0.353 
0.000 
0.914 



sooecs 

RSSIDUAL 



StS^-CP-SlOARES 

10.101 
63.774 



ANftLYSIS CP VfiRIANCE 
IF tSAN-SQOARE 



4 

155 



2.525 
0.411 



F-RfiTIO 
6.138 



P 

0.000 
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Table 13 



Pegression of HOIAV on Schools* and Lessai Background Variables 

(N=160) 

MUmPU: R: .558 SQCIRRED MUmPLE R: .311 

ADJUSTED SQUARED MUlinPIE R: .275 STRNDftRD ERROR OF ESTIMftlE: 0.580 



VRRiaHLE 




SED mBDR 


SID OQEF 


T 


Pf2 iKrtA 




4.450 




0.673 


0.000 


6.609 


0.000 


ARA 


0.552 




0.296 


0.329 


1.866 


0.064 


BRK 


0.153 




0.287 


0.090 


0.535 


0.594 


CAR 


0.960 




0.245. 


0.565 




U« UUU 


GRN 


0.649 




0.328 


0.382 


1.982 


0.049 


GRADEAV 


-0.083 




0.053 


-0.119 


-1.582 


0.116 


ABIIAV 


-0.050 




0.095 


-0.042 


-0.525 


0.601 


BEACK 


-0.004 




0.005 


-0.150 


-0.877 


0.382 


DUMREQ 


-0.171 




0.123 


-0.105 


-1.392 


U«XQO 








mojusis OF vKsims 






SOORCE 




EF 






p 


RBGERESSICN 


22.993 


8 


2.874 




8.530 


0.000 


RESIDUAL 


50.882 


151 


0.337 









*l£ssons were assigned dunnry values, 1 or 0^ for eadi of the variables: ARA (Arnold) , BRK (Bradley) , 
CAR (Carlsberg), (SEN (Grandville) . Scarfx^rough was the "constant." CJoefficients r^rtssent the 
difference in the IIOTAV laean between eac±i school and ScarboroiK^; for exairple, ALiiold exceeds 
Scarborwc^ti's xaean by .55. 

48 



II - J^pendlx 



CSDservaticml Scales for Assessing Higher Order Thinking 
in High Sdiool Scxjial Studies Classes 

For sccdes 1-14/ classes were rated froa 1-5. 1 = »»very inaccurate 
description of this class; 5 ^ ^Very accurate" description of this 
class. Technical notes on the scoring of the scales will be made 
available at the conclusian of the stuc^. 

GENERAL 

1. There was sustained examination of a few topics rather than 
siperficial coverage of many. 

Mastery of hi^ier order challenges requires in-d^yth stuc^ and 
sustai n ed concentration on a limited number of topics or questions, 
lessons that cover a large number of topics give students only a vague 
familiarity or awareness and, thereby, reduce the possibilities for 
thou^tful study. 

2. Ihe lesson displayed substantive coherenoe and continuity. 

Intelligent progress on hi^ier order challenges demands systematic 
inquiry that builds on /.Levant and accurate su!:^tantive knowledge in 
the field and that works toward the logical ee^eJrpaent and integration 
of ideas. In contrast, lessons tliat teach laaterial as imrelated 
fragments of Jaxr-^edge, without pulling them together, xmdennlne such 
inquiry. 

3. Students were given an e^ropriate amount of time to think, that 
is, to prepare responses to questions. 

Thinking takes time, but often recitation, discucsion, and written 
assignments pressure students to make responses before they have had 
enoij^ time to reflect on their responses. 

PrcKioting thcwi^tflilness, therefore, requirer. periods of silence vdiere 
students can ponder the validity of possible responses. 

TEACHER BEHAVIOR 

4. Ihe teadier asked challenging questions and/or structured 
challenging tasks (given the abilily level and pr^".ation of the 
students) . 

By cur definition hi^ier order thinking occurs only ^en students are 
faced with questions or tasks that demand analysis, interpretation, or 
manipulation of information; thac is, non-rcutine mental work. In 
short, stxKJents must be faced with the challenge of how to use prior 
knowledge to gain new knowledge, rather than the task of merely 
retrieving prior kncwlec^e. 

5. Hie teacher carefully considered ejqplanations and reasons for 
ooncl\:isicns. 
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II - i^ppendlx ^ 2 



Bie resolution of hi^ier order challenges often depends not siiply upon 
offering e>5)lanations and reasons^ hut \jpon the quality of e3q)lanations 
or reasoning given to si^^ort conclusions. Rigorous analysis and 
evaluation of reasoning/ therefore, is central to the prcrootion of 
thou^tfulness • 

6* Ifcs teacher pressed individual students to justify or to clarify 
their assertions in a Socratic manner* 

Socratic interchanges probe an individual's level of tmderstanding. 
Siach dialogue presses the student to consider the validity of evidence 
and reasoning/ to suggest alternative perspectives that may othervdse 
escape attention/ and to identic rather rigorcwsly \ghat one does and 
does not knew. Ihese probes usually constitute their own hic^ier ord«j: 
challenges as th^ facilitate develqproent of knowledge on the topic in 
question. 

7. Ihe teacher encourarpd students to generate original and 
vmconventional ideas, e55)lanaidQnS/ or solutions to problems. 

Hi^er order challenges can require creative/ intuitive insic^ts and 
altermtive perspectives that diverge from convCTtional knowledge or 
froKi expected lines of reasoning. Pr:«noting this sort of mental 
flexibility will help students to cope not only with ill-structurad 
problems that may e)5)licitly invite creativity/ but also with well- 
strvictureci ones that may requi. e xanoonvsntianal thinking. 

8. 2he teacher showed an awareness that not all assertions emanating 
frcm authoritative scw:;:ces are absolxite or certain. 

Cariosity and sto^)ticiFm about the creation of kncwledge and the nature 
of truth are di£:yosi'ti<^riti of thc«)^tfulness that help to sustain hi^ier 
order thinking, ioz- these dispositions to he promoted/ axithoritative 
sources must not always be accepted xmcritically. • Instead/ students 
must be mada aware that vihat itay pass as final/ conclusive knowledge 
may often be prcfclematic and subject to future revision and 
develcpnent. 

9. Students • ptirsonal ej^irience (where relevant) was Integrated into 
the lesson. 

Student engagement with and cosrprehension of a problem can often be 
enhanced if inquiry on the problem is related to students* actual life 
esq^eriences. 



10. The teacher was a model of thou^t fulness. 

To help students succeed with higher order challenges/ teachers 
themselves must model thou^tfulness as they teach. Of course/ a 
thouj^tful tea^iher would demonstrate many of the behaviors described 
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II - Afpendix - 3 

above/ but this scale is intended to capture a clxister of additional 
characteristics likely to be found in any thoo^tful person. Key 
indicators include shcxdng interest in students' ideas and in 
alternative e^pproaches to problems; shewing how he/she thc^t throu^ 
a problem (rather than only the final answer); and aciknowledgijig the 
difficulty of gadning a definitive imderstanding of prciDlenatic topics. 



SIWDENT BEHAVrCR 

11. Students offered esgplanations and reasons for their conclusions, 

The answers or solutions to hi^ier order challenges are rarely self- 
evident. Iheir validity often rests on the quality of explanation or 
reasons given to si^^xDrt them. Therefore/ beyond offering answers^ 
students raost also be able to produce e^qplanations and reasons to 
seaport their conclusions. 

12. Students generated original and unconventional ideas ejqjlanations/ 
hypotheses or sol^ttions to problems. 

This is a ocmterpart to scale #8. Hic^ier order thirOcing is itore 
evident in classroceis in vMch teaciiers jwt only encourage original 
responses frcxi students, but vSaere students also simply them. 

13. students assumed the roles of questioner and critic. 

As students work toward answers for hi^ier order cJiallei^geS/ they are 
laore likely to succeed if they have opportunities to ask questions 
about and to criticize ptxposed approaches and answers. In contrast/ 
if students act prjbmriJLy as passive recipients of information/ they 
are riot likely to develop the level of understanding needed to carry 
out useful analyses and interpretations. 

14. stu d ent contributions were articulate/ germane to the topic and 
connected to prior discussion. 

In a formal sense/ students may respond actively with assertions/ 
e35)lanationS/ questions/ and creative ideaS/ but these may be 
lnia±iculate and irrelevant to the topic beii^ examined. iJust as scale 
#2 requires substantive coherence and continuity in the lesson as a 
vihole, student j^articipation cannot be consideced thou^tful unless it 
is reasonably articulate and relevant. 

15 c, VBiat prc^xartion of students participated verbally in the lesson? 

To solve hitler order challenges/ students itiust actively use their 
minds. This cannot be observed directly/ but one indicator is the 
extent to ^Adch they speak in class about the subject being examined. 
Biat this, the more students vdio produce discourse on a topic the 
greater the likelihood of hi^ier order thinking by the class as a 
v*iole. 
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II - .^5)endix - 4 



16. What p roportion of tiiive did students spend er^ged in thou^tful 
discx)urse with one another? 

Because of the iitportanoe of dialogue in prcKootir^ hi^ier order 
thinking, yet the logistical difficult of the teacher respondir^ to 
the irjdividual ideas of each student, it can be tiseful for students to 
talk with one another to test and refine their ideas. Die amount of 
thouc^tful discourse among students can, therefore, be seen as one 
indicator of thou^tful discourse in the class as a vSiole. 

17. What proportion of students shewed genuine involvement in the 
topics discussed? 

To make pr o gress on hi^ier order challenges, students must give their 
attention, oonoentraticn, and mental effort. We summarize these 
dimensions of involvement as student engagement. One indicator is 
observable verbal rarticdpation? but it is also pc«sible to beoosas 
intensely involved vMle listening. 3his scale assesses the ejctent of 
student engagement based on cues that extend beyond speaking. 
Principal indicators lixxLude raising hands, attentiveness manifested by 
faci2d e)?>res&icn and body-language, interrvKJtions in the discussion to 
raise a substaxitive point or question, the length of stiident responses. 
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ECT inaW S ASEBCIS OF !IEAaiEBS« ^IHINKINS 

ifflODT HCKxcnis snsxins' miKKiHs 

Joe GteekD 

Ii niODOCTICK : 

A praninent theme of educational reform in the 1980»s is to 
develop students' "hi^ier order" and "critical" thinking* Ihis has 
been reflected in numerals: a) national reports advocating 
increased estjliasis on thinking in schools (Adler^ 1982; Bayer/ 
1983; CJollege Board/ 1983; Sizer, 1984; etc.); b) scholarly works 
that attenpt to concqptualize thinking; c) omriculum naterials 
developed to help teadi^rs pronote students' thinking; d) workshops 
and programs on thinking sponsored by school districts and 
professional organizations throu^iout the country; and, e) national 
and state efforts to incorporate test items requiring hi^ier level 
thinking on achievement tests and other standardized assessment 
instmments. 

VSiat is striJdngly absent in the literature and in 
programmatic reform efforts is an attenpt to tmderstand and learn 
fr\:»i the work of practitioners. Ohis my be due in part to the 
blfiak portrait of actual classroom practice painted in the 
literature (CXiban/ 1984; Goodlad/ 1984; Htoether and AWljrand, 1969; 
Berrone, 1985; Sirotnik, 1983). Nonetheless, we know that scoe 
teachers do challei^ students intellectually (lic^tfoot/ 1983; 
Sizer, 1984; Swartz, 1987) . If we are to understand hew to prcroote 
thinking in the classroou/ we need to learn more about how teachers 
think about their work. Over the past ten years research has 
docuiaented that teachers • thou^its and beliefs play a crucial 
inediating role in their classrocm practice (dark & Peterson, 1987; 
Peter.5OTi, Fennema, & Carpenter, 1987; Shavelson & Stem, 1981; 
Schuliian, 1987; Zeictmer & Teitelbaum, 1982). As an ej^Jloratory 
effort: ws ccBppare the thou^t of teachers who c^te consistently 
proQofbB thinking in their classrocrns with the thou^t of teachers 
\*o do so less consistently. Should important differences emerge 
in the thinking of the two groi^^s, these differences may provide 
clues to influencing classroon practice throu^ changii^ teachers* 
thinking about practice. However/ if only siinilarities are found 
in the thinking of the two groapa then no clues can be offered. 

A sanple of 10 hi^ school teachers were selected frcm a total 
pool of 15 teachers frcm 5 secondary schools (3 teachers per 
school). Each teadier was observed on 9 occasions.^ Three class 
observations took place during eacih of the 3, 4-^y school-site 
visits spiread over the course of the 1986-37 academic year (i.e., 
fall/ winter, spring) . In addition, at least 2 hours during each 
of the three site-visits was spent in interview with each teacher 
to probe tlieir thinking aboJjit teachir^, students, colleagues, etc. 
Ihe teachei's are inenibers of social studies d^)arbnents that have 
made conscious efforts and progress at fostering students' 
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thinking. The 5 schools were selected throu^ a nation-wide search 
in \ijhic3i 60 pranising social studies d^artioents were nccdnated. 
Through extended phone conversations with the priricipal, department 
head, and one or two teachers, and a one-<3ay site visit, jtidgnients 
wers made regardii^ the degree to \ghich each d^artment fulfilled 
criteria specified in the ncmination request amwuncesnent. 
Criteria inclvjded a d^artment-wide en^-jsis en thinking as 
witnessed in the practices of department menibers and in 
conversaitions with thsm, and that thinking was enjiiasized in 
required courses and in courses with a preponderance of 
lew-achieving, Icw-SES, or minority students. The teachers in the 
pool of 15 are considered by the dspartanent diairs to be among the 
best in their respective departments at eitjSiasizing thinking with 
students. 

The 5 teachers with the hi^iest percentage of lessons 
receiving a score of "4" or "3" on each of the 6 cfaservationa]. 
scal^ representing the minimi regiiirements of a thoughtful 
disjccwrse lesson were designated "high scorerSi"^ see Newnam 
(this volume) vMch provides a rationale for and description of the 
cbservational scales used to measure the thouctfitfulness of 
classroan disccurse. Z.ie 5 teadiers with the lowest percentage of 
lessens receiving a moderately hi^ score of 3 or more cr. ^ach of 
the 6 cbservational scales were designated "lower scorer^ ^ The 
remaining or "middle" 5 teachers in the pool of 15 vere excltided 
from the analysis. 



Ipive of the 15 teachers were observed cai ei^t rather than 
nine occasions due to personal illness, a snow day, etc. 

^Because all 6 scales are viewed as i^ecessary caiponents for 
thoughtful discourse, a hi^ score (i.e., "4" or "5") was reqalred 
on each scale before a teacher's lesson was to be considered 
outstanding, l^ing teacher performance on the mean score of a 
lesson across all six scales was not used. The reason for this is 
that a low score of "2" or "3" on a few scales caribinad with 5's on 
the remaining scales could result in a hi^ier mean score across the 
six scales than a lesson receivir^ all 4 's— 'leading to the possible 
exclusion of the latter as an exeirplary lesson. 

^ixie to the skewedness of the sairple, the lower scoring gnxq? 
is considered less-cwtstanding rather than non-cutstandiiif with 
respect to the hi^ scorers. Therefore selection of lower scorers 
was based the lowest percentage of good lessons (i.e. , a score 
of "3" or more on each of the 6 scales) rather ';^an the hi^iest 
percentage of poor lessons. it should be noted that the third 
observational scale (i.e., teacher gave students time to think) 
required a score of "4" or more rather than "3" or more, as 
otherwise all tut 18 of the 135 lessons would qualify as moderately 
good lessOTs on this dimension of teachers' practice. 
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Teachers were unaware of the specifics of the study's research 
agenda. 2hey did not knew that two grcqps of teachers would ba 
conpared on various dimensions of their thinking about teachir^, 
students, colleagues, etc. Nor were teachers aware of the 
observational scales used to measure dimensions of thou^tfulness 
in their lessens. Researchers summarizui teacher responses did 
not knew vhich teachers ftcm the pool of 15 would ocnprise the two 
gxxxps of hi^ and lower scorers until statistical analyses were 
performed months ed^ter the data was gathered. 

Five areas of teachers' beliefs and theories were explored: 1) 
teachers' conceptions of thinking; 2) the relative iitportance given 
to thinking as an instructional goal; 3) their views ai the issue 
of depth vs breadth of content coverage; 4) their perceptions of 
organizational barriers to prcroofcing thirfdng in dassrocnis; and, 
5) their v:lews of students. Our selection of these 5 areas was 
based in part on previous research. 

Previews research indicates, for exanj/le, that teachers 
possess a variety of theories and beliefs about instruction (e.g. , 
ideas on learning, reading, classrocm vemag&njnt, student 
motivation, etc.), and that these theories and beliefs help guide 
their instructional efforts (Clark & Peterson, 1987; Marland, 1977; 
Muribi-, 1983). . Indeed, it is difficult to imagine a teacher 
pursuing an i.nstructional goal without seme image, set of ideas, or 
conception of the goal being pursued. We were therefore curicAis to 
see if hi^ scorers: a) possessed different anchor more elaborate 
conc^jticns of thinking than lower scorers; and, b) placed greater 
enphasis on thinking as an instructional goal than lower scorers. 

TLoning to the issue of depth vs breadth of content coverage, 
at one end of the spectrum are teachers vlho rigorously pursue 
breadth of coverage, galloping their leamir^ herd over numerous 
content pastures at a mind-numbing pace and reducing students' 
educational es^jerience to sinplistic menorlsation of names, dates, 
places, events, etc. At the other end of the spectrum are teachers 
v4io rigorously pursue depth of coverage, dialling their learning 
herd to a few content pastures and reducing students' educational 
eJ5)erience to detailed understanding of a relatively narrow range 
of human experience. Is either s^proach more liJoely to pranote the 
develcpaent of students' thinking? Tbe conceptual literature on 
thinking highlights the inportance of thinking skills, 
dispositions, and content understanding for effective thinking 
perfonnanoe (Newmann, 1988a; Siegel, 1988) . Presumably, time spent 
developing students' thinking skills and dispositions, and their 
understanding of the content under study, will reduce time for 
ccaitent coverage. Some educators have in fact argued that there 
must be a movement away fran massive coverage if students are to 
develop an ability to thinking de^ly and effectively about tcpics 
and issues (Newmann, 1988b; Sizer, 1984). How do teachers 
attorptlng to pranote students? thihking tackle this dilenana? We 
were curicus to see if hi^ and lower scorers addressed or 
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attainted to resolve the dilemma of d^?th vs breadth of coverage 
differently. 

Ihe pronotion of thinking also seems related to organizational 
features of teadiers' v^aric setting. For exanple, large class 
size, large total student loads, relatively short 45 minute class 
periocJs may severely constrain teachers' instructional efforts 
(Pcwell et al., 1986; Sizer 1984). Die adulteration of academic 
tasks thrcuc^ the process of "n^otiation" between teacher and 
students (Mtifeil, 1983) may, in part, be due to these debilitating 
organizational features. Indeed, one respected observer of the 
classrocm has argued that the developraent of students' hi^ier orxJer 
thinking and organization features of the traditional classrocm are 
incccpatible if not imituaUy exclusive (Cuban, 1984b). Scroe 
research su^ests that fnastration over working conditions is a 
primary reason teachers leave the profession (Goodlad, 1984) . We 
were curious to see if hi^ and lowsr scorers perceived or 
attenpted to modify various organizational banriers differently. 
J/Sore ^)ecifically, do hi^ scorers posses55 laore effective 
strategies for dealiiig with these barriers? 

Finally, we looked at teachers' peroept'lons of students. Much 
research has documented the powerful inpact teachers' perceptions 
of students have on the instructional process, inclxiding the number 
and kind of classrocm interactions between teacher and student 
(King, 1980; Page, 1984; Peterson & Barger, 1984), the types of 
instructional goals pursued and the way classroom behavior is 
managed (Brofphy & Rohrkeirpter, 1981), axxi teachers' esgjectations of 
student performance (Good, 1987). We wanted to see if hi^ 
scorers' perceptions of students are in any ways different frm the 
perceptions of students ^eld by lower scorers. 

CBMBKRISCK OF HLGR NUD lOHER SOCRBB CN BRCSEBCKISD CHHaCISRIgriCg 
AND CRSANIZaiEIOW DIHEM5iaNS OF THEIR WSCK z 

Before presenting the findings, a brief ccmparison of tht. two 
groips is provided on selected background characteristics and 
organizational dimensions (see T&ble 1) . Such a ccofrparison may 
respond to suggestions frcci previous research that factors of class 
size, ethnic background, prior achievement, etc., affect learning 
cwtccmes and instoictional practices. Do hi^ scorers in our 
sanple have a more favorable teaching situation with respect to the 
above variables? If major differences betweei the two groups fail 
to emerge on these variables then e:>5)lanations for observed 
differences in the instructional practices of the two groi5)s will 
need to rely on other sources. 

Four of five teachers in both the hi^ and lower scorirg 
gra^ are male. Ihis percentage closely parallels national 
statistics in viiich wobsn constitute only about 26% of school 
social studies teachers (Rutter, 1985) . The hi^ scorers averaged 
14.6 years of social studies teaching e>$)erience, v^Je the mean 
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for the lower scorers was 17.0. 



Hcwever, ejqjerienoe varied 



griKitly within each grcxp; 2 to 21 years for the high and 3 to 25 
years for tlie lower scorers. The amount of fontal education is 
ccnparable for the two grotps of teacJiers. Most of the hi^ 
scorers (4) hacve a Masters' degree plus additional credits, vMle 
the raDaainlng meancfaer has a Masters' degree. Among the lower 
scorers, l has a Doctorate, 2 possess a Masters' plus adaitional 
credits, and 2 earned a Masters' degree. Jcb satisfaction is also 
ocoparable, as aost teadiers (4) in both groi^js state they are 
satisfied vdth their job "roost" of the time vdiile the reaaining 
mentoer from each grcup derives satisfaction only "half" of the 
time. 

TJiree of 5 schools are represented by teachers in the hi^ 
scoring grasp and 4 of 5 schools in the lower scoring groi?), and 2 
schools have teachers \*o a^^jeared in each groc?>. with reject to 
subject matter, 3 high scorers tau^t a history class while the 
other 2 taiJ^t social science courses (i.e., eooncndLcs and 
psychology) . within the lower scoring grot?), 4 taught history and 
1 a social studies course (i.e., American government, or "Civics"). 
A sizable difference between the two grou^js emerges with respect to 
the nuntoer of teaching periods per day. High scorers averaged 4.6 
teaching periods per day whereas the lower scorers avera^ one 
less teaching period, or 3.6. Cla^ size however was about the 
same, 28.6 for the hi^ and 26.4 students for the lower scoring 
grasp. Because of the additional class assignment hi^ scorers' 
total student load was substantieilly hi^ier; 131.8 students 
ccaroared to only 97.6 for the lower scorers. Qie ramiber of course 
pr^sarations was essentially the same, 2.2 for hi^ scorers and 2.4 
for Icwer scorers. 

Turning to student class ccEtpositions, most of hi^ scoisars 
(4) woriced with students of diverse or low achievement levels 
vhereas a majority of the lower scorers (3) worked with hi^ 
achieving students. ^ Both grocps had similar Hispanic 
carpositions; 11% for the hi^ and 12.8% for the lower scoring 
teachers. Hi^ier scorers had more l^ian students than lower 
scorers, 5% to 1.2%, tut fever blacOcs, 12.6% to 28.8%. However, 
the dii^jarity between hi^ and lower scorers regardirg percentage 
of black students should not be overen^hasized as variance was 



^At each school we requested for observation three types of 
classes: a) primarily lew achieving students; b) a diverse rar^ of 
student achievers; and, c) the class eadiibiting the most hi^ier- 
order thinking (v4iich usually cctprised hi^ achievers). Prior 
social studies achievement levels of scudents were determined by 
the d^artment head and later confirmed by the teachers thauselves. 
Student achievement data on teachers' classes pertains only to the 
particajlar class researchers observed for the observation portion 
of the study design. Data was not collected on prior achievement, 
ethnic, or racial conpositions of teachers' total student load. 
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large within each grcn^; i.e., tram 0% to 37% for hi^ ana 0% to 
74% for Icwer scoring tt\achers. In addition, roost lower scorii^ 
teachers (3) had less tlian 3% blacilc students' in their respective 
classes. 

To sujunarize tram the above data, siinilarities e»36ed 
differences between the two c|roi?3s of teachers. And where 
stjbstantial differences do exist, lower scorers tend to have the 
more favorable situation (e.g. , fewer # of students, fewer teaching 
periods, more hi^ achieving students, etc.). Biis cbservation, 
hcwever, slwuld hot be tafeen to iirply that assigning teachers a 
larger student loaci, roore lew achievers, or roore class periods per 
6py will increase the level of classrocm thou^tftOness. Ihis 
finding nay siaply be an artifact of the small sanjjle size. But if 
this rgsresents a real difference, it may be interpreted to rosan 
that teachers' instructional approaches can ov^erride otherwise 
constraining organizational dimensions of their work. Because this 
finding runs counter to all of the recoraapndations for teacher 
professional develcpnent and inprovement, future research should 
3<eep this interesting ancroaly in mind. 

A nuniber of questions were used to identify teachers' 
instructional goals. The findings traa any single question may not 
be ccatpelling vdien taken in isolation, but as a grot^) the questions 
generate findings that hi^ scorers place greater eiiphasis on 
thinking than tl^ir lower scorii^ counterparts. 

Biou^ teachers frcm both the high and lower scoring graaps 
all state that helping students beccape better thinkers is very 
iirportant for their feelings of success, \dien asked to identify 
their highest priority goal for social studies instruction and the 
goals that focus their lesson planning differences between the two 
groups begin to eioerge.s with respect to their hi^iest priority 
goal, "critical thinking and problem solving" was given greater 
ernsghasis by hi^ scorers; that is, roost hi^ scoring teachers (4) 
vs a f ew lower scorers (2) cited thinking ani problem solving as 
their hi^iest goal (see Table 2). That thinking and 
prdalem-solving was less eitEhasized bv lower scorers could 
theoretically prove to be an insignificant" finding, as thinkii^ can 
play praninent role in the pursuit of other goals (e.g., 
"teacSiing facts, conc^jts, and th^ries"; "developii^ discussion 
skills"; etc.). However, the above difference between the two 
grcn^ gains in iitportance vAiSXi one also considers that all hi^ 
scorers (5) fcut only scsme of the lo^rer scorers (3) cited "thinking 
and prcblem solving" as one of the many goals that focus their 



^teachers were asked to select their hi^iest goal and the 
goals that focus their planning frcm a list of potential goals (see 
Table 2). 
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lesson planning efforts (see Table 3). Note that teachers here 
wera encouraged to icJentify an/ and all goals that focus their 
lessOT plannfiig, yet one finds a few lower scorers not mentioning 
thinking as a planning goal. 

m addition to the above findijngs, inconsistencies eaaerge in 
the responses of Icwer tut not hi^ scoring teachers. A few of the 
lower scorers (2) failed to list their stated hi^iest priority goal 
as one of the goals that focus their lesson planning. No such 
onissicn occurred among hi^ scorers. For example, one lower 
scorer idratified teaching students about "past and present 
prcbleas and issues faced by the U.S. and World" as her hi^iest 
coal/ yet the goals focusing her lesson planning vjere to "teach 
students constructive social Va'^ues and foster responsible 
citizenship" / and "teach students facts, conc^rtzs, and theories 
central to understanding history and the social scieiK?es." 

Inconsistencies also arose among the same two lo^^ scorers 
with respect to their hi^iest priority goal; that is, their hi^est 
priority goal changed from visit 1 to visit 3. For exanple, one 
lower scorer identified "teaching students critical thinking and 
problem solving" during the fall (visit 1) as her hi^iest goal yet 
in the spring (visit 3) replaced it with "teaching students 
constructive social values and responsible citizenship." Ihere is 
no indicaticai or reason to believe that tiie instructional approach 
of either teacher had changed during the year- 

In another query of teaa>iers» goaJs we asJosd hi^ eoyi lower 
scorers if ej^posing students to subject matter content is, in 
general, ••more", "eqivdly", or "less" interesting to them as 
develc^ing students' thou^t and reasoning processes. All of the 
high scorers find ej^xDsing students to subject matter content less 
interesting cccpared to only 1 among lower scorers. Most of the 
Icwer scorers (3) find ccaitent ejqposure ecpally interr>sting \Mle 
one finds it, in fack, more interesting.^ Biese findii^ (see 
Table 4) are consistent \d±h findings cited above in vMch hi^ 
scorers place greater errjiiasis on thinking as a highest priority 
goal and as a goal focusing their lesson planning. 

Additional support for the differing enjiiases placed on 
thinking by the two groi?>s cai be culled fixia teaoiers' responses 
to the foHowirig general prcbe: 'TShat in particular gives you 
satisfaction as a teacher?" Teachers' rajspons^s can 1» groaped 
into the following four kinds or categories of satisfaction: a) 
seeij^ students thinking; b) seeing students responding; c) working 



%xgever, ^*en teachers are asked if e)q)osing students to 
.ontent vs developing students' thinking is more or less 
"difficult" (rather' than "interestii^') , both groins (4 of 5) 
agr^ that is was "more" difficult to develop students' thinking. 
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vdth colleagues; and, d) constructing lesson plans (see Table 5) . 
With respect to the latter two categories (i.e., world! g with 
colleagues and constructing lessons), the responses of hi^ and 
Icwer scorers are very similar. One teacher from each graap made 
reference to lesson planning as satisfying, vMle 3 hi^ and 2 
IcwBT scorers cited their work with colleagues as satis^ing. 

Htwever, Inportant differences e^^jeared between the two groips 
regarding the other satisfaction categories (i.e., students 
thinking and students responding). All of the hi^ scorers (5) 
cite as satisfying student behaviors one would associate with 
thinking (e.g., "seeing students start to make connections", 
"seeing students gain a more precise understanding", "students 
wrestling with values and making links", "leading students to grasp 
concepts/skills", "teaching students to generalize frm data", 
etc.), vtoeas only 1 Icwer scoring teacher cited a satisfaction 
one mi^t associate with thinking—and that rather marginally 
(i.e. . "touching a young mind") . All of the Icwer scorers (5) , 
howc/er, say they derive satisfaction v*en students are responding 
in class (e.g., »v4ien bcudents show interest", "vfiien students are 
generally responsive", "the responses of students", etc.), whereas 
only 1 hic^ scorer made a ccsninent indicating satisfaction vSien 
students are re^xjnding (i.e. , "students ccranenting") . 

Die rather profound differences between the two qjxnjps 
regarding student behaviors they fiM satisfying may sinply be a 
matter of semantics; -Uiat is, Icwer scorers may actually mean 
thinking-type student behaviors vSien they say they derive 
satisfacticai from responding-type behaviors. However, even 
assuming this were the case (xMch is not at all clear) , responses 
indicate that hic^ scorers identify with greater specificity the 
kinds of student behavior they find exciting and satis^ing, and 
that these behaviors are t^ically associated with acts of 
thinking. 

Similar findings emerged between the two groc^xs v^ben teachers 
were asked cei an qpen-ended question to identic the kinds of 
thinking tasks less receptive students are likely to resist. 
Ihou^ responses were quite varied—both within and between 
grams— -tt» tasks that hi^ scorers nentitsn are sore areadily 
identifiable as involving hi^ier order thinkis>g, again suggesting 
that hi^ scoresra place greater eicphasig on thinking as an 
instructional goal. Ohe follcwing thirfdng tasks are cited by hi^ 
scorers as most frequently resisted; giving reasons for statements, 
givii^ opini'^, doing a critical analysis, defending one's 
viewpoint, engaging in value reflection, si^^jlying metaphors, using 
precise language, dealing with abstractions, essay writirg, and 
work requiring extended effort. Lower scorers on the other hand 
cite activities that for the most part may or may not involve 
thinking: doing homework, participating in discussion, reading in 
class, doing vocabulary assignments, anything beyond rote memory, 
anything without st^ by stqp instructions, essay writii^, work 
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that is too difficailt, long readings, and dealing with 
abstractions • 

Tne above differences in goal orientation between hi^ and 
Icswer scoring teachers are further revealed during an interview 
session and on a ^lestionnaire item \dien we asted teac^iers to 
discuss their instructional goeLL(s). Hi^ scorers' responses are 
longer, more elaborate, and pi ace greater esoo^iiasis on the 
developooent of students' thinking. 

In addition, one senses that hi^ scorers see their work 
directed toward a broader, more far-ranging mission that lower 
scorers. Besides producixig inprovements in stidents' lOKwledge 
structures and thiiMng abilities, high scorers hope thc^t stcdents 
will emerge frcci their classes transfonned in more fa2>xeachir» 
ways. For instance, consider the way one hi^ scorer, }to^/ 
describes his mission: 

CJontent is a vehicle to teaching critical 
thinking, thcwj^ there are certain things you 
want them to know vftien they leave econcciics. 
No matter ^Aiat subject, you can get students to 
think abcwt big issues... I'd like kids to 
always be questioning, to always te probing. 
You should fidways be on the edge, nev;er 
ocBttfortable, no itatter hew well you've digested 
the material... don't take anything at face 
valiie...I never saw a lescon vAiere I went tn 
school vftiere they didn't know the 'answer' at 
the end of the trail... I want kids to be able 
to say, 'Hey, I'm a person, I can thnik! 
That's vteit I'm all about. I'm si:5pose to 
think. ' To think is a helleva lot better than 
to know. To '^kncw' can be a dampening 
e^^jerience sometimes because you're not going 
to probe anymore, you've &qper)ded yourself.^ 

Another hi^ scorer, Hilary, also hi^i^ts critical thinking, 
placing special enjiiasis on perspective-taking as an essential 



7a11 hi^ scorers' names will begin with the letter "h" as in 
"hi^", while lower scorers' names will begin with "1" as in 
"lower". 

^teachers' responses are taken frm questionnaires and 
fdllOfMip interviews. Socae quotes are direct, vMle others are 
quoted fron researcher's notes. Researchers' notes capture the 
ideas es^pressed but may not represent the teachers' cannaents 
verbatim. Wherever possible effdrts were made to return to the 
audio-tapes to check for response accuracy. 
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aspects of critical thlrfdiig abcxit social and personal values. She 
states: 

Uhtil you can begin to at least tenporarily put 
yourself in the other guy's shoes, even if you 
aren't going to end yjg there, you caniK^t evolve 
a set of social valuer> that are good for you or 
for society... It's ixcportant for students to be 
able to st^ into the perspective of another as 
it enables them to better understand the total 
situation, and to defend their own position if 
they still maintain it after perspective 
taking. . .The roots of prejudice and 
discrimination may lie in an inabilxty to see 
other perspectives. . . in an over-progr a mmed, 
media- centered world, decisions and soliitions 
to problems are too often 'made' for people. 
Students develqp a pattern of letting soroeone 
else do their thinking. Ihere is a need to 
teach thinking so that kids will think for 
thGmselves...I try to get students to question 
and formulate positions. . .to be able to e)qplain 
and si^jport ideas. . .to see that question asking 
is a sign of intelligence not ignorance. . .to 
get them to knew that they don't know and want 
to find out. 

Exc^ for one member, lower scorers ' responses to the 
question of instructioncd goals were generally much shorter and 
lacOced the^ impassioned elaboration of hic^ scorers—regardless of 
the goal cited. Scanae lower scorers siitply parroted from the list 
of goals we offered as exaitples on the questionnaire. Lew scorers' 
responses include the following: 

Critical thinking and problem solving. 
Developing constructive social values. I have 
beccwie far more subject oriented. I started 
teaching in the late 60»s and 70's. I find 
there is a far greater need to provide students 
with a sound data and conc^ base. [Lisa] . 

Teaching facts, corcepts, and theories of the 
social sciences. Our district is enjiiasizing 
the develqpment of discussion skills, public 
speaking, and -things of that nature. Wfe 
emphasize discussion within a convergent 
ftamework in social studies. . . I take my 
directions more or less from the 
district. . .they tell us vtet it is they expect 
from us... now they are moving toward hi^ier 
level thinking in the curriculima and getting 
beyond the literal level... In my curriculum 
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ri^t new it's not in there but in scane 
curriculxiras it is being inpleiaented. . .1 think 
it is their responsibility as an administrator 
to make kncwn to me viiat is being tau^t or 
stould be tauc^t. [Lloyd]« 

To suramurize, hi^ scorers are more likely to identify 
thirJdng as their hi^iest priority goal and as a goal focusing 
their planning. Also, hi^ scorers, vhen asked \Aiat gives them 
satisfaction as a teacher, are likely to provide more detailed 
descriptions than Icwer scorers of the kinds of student behavior 
they find satisfying, and that these descriptions are more likely 
to be associated with acts of thinking. UhliJce most of the lower 
scorers \*o find exposing students to subject matter content 
equally or more interesting than developing students' thinking and 
reasoning, hi^ scorers lananimously prefer to develop students' 
thinking. In addition, contradictions emerged in the responses of 
lower scoring teachers but not hi^ scorers; that is, the stated 
hi^iest goal of two Icwer scorers changed over a 6 month time 
period withcut e3?)lanation, and these same two feiiled to mention 
their highest priority goal airtong the goals that focus their lesson 
planning. Finally, the goal statements of hi^ scorers are imch 
lengthier and more detailed, focus more an thinking, and suggest 
that hi^ scorers desire to affect students in far-ranging, 
charactei^'liice vays. When ccwbined, the findings frcm the above 
researcai probes form a montage that illvistrates a more consistent 
instructional eanophasis on thinking among hi^ scorers ccaipared to 
lower scorers. 

BftHkUflg TO TOE HCMariCW OF THINKINS : 

As in the goals section above, a variety' of questions were 
xased to describe teachers' perc^ions of instructional an^ 
organizational barriers to proanoting students' thinking. Again, 
the sum of the parts paint a clearer portrait than does any single 
part. In this portrait, however, similarities outwei^ differeri:es 
between the two grotps of teachers. 

Teachers were asked to indicate on a scale of 1 to 5 
(extremely negative to extremely positive) the effect various 
organisational barriers have on their Glasssrccsa efforts to prr-grvots 
students' thinking (see Table 6). Both grocps indicated that 
"large class size" and "large student load" had the most negative 
iitpact on their instructional efforts, with the mean score for both 
grwqps well belcw 2.0. Ihe only other organizational feature that 
was viewed by both groi:?)s as having a negative effect on the 
promoticxi of students' thinking is "students having to take too 
many courses." Ohe only or^mizaticmL feature to receive a 
positive rating (i.e., above 3.0) i3y both groips of teachers was 
"4-5 class sessions per \^^ek". The two grot^^s displayed a sizeable 
difference in their ratings only with respect to their attitudes 
about "ability grca:5)ing". Hi^ scorers find ability groapixQ 



HI - 12 



scxnewhat detriiaental to their instructional efforts {x==2.8), vMle 
lower scorers find ability grouping scroevAiat beneficial (Xr=3.8) . 

Simlar findings emerge ^en teachers were asked to identify 
the 3 barriers that itost Irtiibit their attenpts to prcBnote students 
thinking (see Table 7) . Many high scorers (3) and itost lower 
scorers (4) cited "large student load" as one of the three itost 
inhibiting barriers* Both gnx^js (3 teachers each) also cited 
"large class size. " Die data suggests that teachers are not 
frustrated by the nuinber of classes th^ must teach, as only l 
teacher in the entire sanple of 10 identified "number of classes" 
as detriiaental to their instructional efforts. Rather teacher 
concern focuses on the large number of students they inust address 
in each class and over tha course of the day. Aside frcm the issue 
of student load, teachers' responses reveal great diversity in the 
barriers they find most inhibiting. ^ 

!Ihe nost inportant finding to emerge from teachers' thou^ts 
on i^tcriers may not be \^t they identify, but rather how little 
they have been able to reduce the impact of these barriers. Once 
teachers identified the three most detrimental barriers to their 
dassrocci efforts, we asked them yhat, if anything, they had tried 
to do to rediioe the effect of these barriers and with v*iat results. 
Of the 15 bs^iers identified by each groap (i.e., 3 from each 
teacher), hit^i scorers' responses indicate that no meaningful 
change or inprovement had taken place with respect to 12 of the 
barrieirs, vAiile lower scorers' responses indicate no meaningful 
change with respect to 11 of 15 barriers. Some teachers shared 
with us their feeble attertpts to reduce the iitpact of these 
barriers. For e:cairple, with respect to "large class size" one 
teacher makes "a special effort to incorporate non-volunteers" by 
calling on them during discussion, or with respect to the barrier 
of "large total student load" another teacher gives an oocasioncil 
oral rather than written quiz to cut down on the eonaofunt of paper 
correction time. 

TtiB few instances vdiere teachers did achieve sate positive 
effect on barriers are not dramatic. For exaitple, one hi^ scorer 
was able to reduce his student load by securing a curriculum 
development position for one period a day, effectively reducing his 
class assignment load frcm 5^ to 4. However, this is only a one 
year ^c5)ointment and only open to' %e teacher in the district per 
year. At another school a lower scorer has reduced his student 
load during class, discussions by occasionally having a colleague 
take half of the students to another classroom for discussion. Of 
course, the net effect is to increase the student load of the 



^Pressure to cover course content emerges as an ittportant 
concern of hi^ scorers. This barrier, hcwever, v/ill receive 
separate attention in the section on depth vs breadth of content 
coverage. 
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obliging colleague, in only one situation was a teacher able to 
significantly reduce student load, and this by dropping to 
half-time enplqyment vMch gave her half a load. Needless to say, 
this is not an option available to most teachers. 

One cannot fail to notice that all of the teachers' efforts to 
reduce the iapact of orgamizaticaial barriers were atteapted within 
the parameters defined by the present school structure? that is, 
teachers did not atteapt to modify strtjctural aspects the school 
institution. We did not fonnally collect data on the reasons for 
this situation, but based 1:5x31 teachers' responses to a related 
question it seems that teacheis are extreioely resigned and 
pessimistic about the liJcelihood of positive change in the 
organizational features of schools. 

Teachers were also given the opportunity to inagine a nore 
ideal teaching situation. Ihey were asked to identify 3 "wishes" 
they would make to aid them in their efforts to praoote students' 
thinking. Again, the oonoem over student load was dominant as 4 
of 5 teachers in both the hi^ and lower scoring groi^js requested 
fewer students as one of their three wishes. Aside fron the 
problem of student load, the hi^ and lower scorir^ grcqps have 
different wishes. Little consensus occurred within groups either 
(see Table 8) . 

Turning fron teachers' percqpticais of various organizational 
barriers to their perceptions of d^artment colleagues, both hi^ 
and Icwer scorers unanimously agree that their colleagues suqc^rt 
rather than undermine their efforts to eitfiiasize thinking with 
students. with respect to administrative authorities (i.e., 
d^artment head, principal, si:5)erlntentaent, etc.,), nost of the 
hi^ (4) and lower (4) scorers feel free to diverge fron the 
conception and set of practices the above authorities espouse for 
prcraoting students' thinking. Ihis suggests that teachers do not 
perceive administrators as undermining their instructional efforts. 

As for testing as a potenticil barrier, all hi^ scorers (5) 
and most lower scorers (4) agree that departnental, school-wide, 
and district tests have "no effect" on their efforts to prcroote 
students' thinking In the dassrocan. Many of the "no effect" 
responses are due to the fact that 3 of 5 schools in the sample do 



^°Por exanple, teachers were asked to identify 3 "wishes" they 
wculd make to aid them in their efforts to prcaaote students' 
thinking. Hhey were then asked to e:^lain vdxat would have to 
happen in their school for the wish to be fulfilled over the next 5 
to 10 years. Typical teacher responses were: ". . .the likelllraod of 
this is nil", "...we have had no reduction In 20 years", 
"...nothing is going to happen", "...little will chaise", 
"...chances; 1 in a million", etc. For details of teachers' wishes 
see Table 8 belcvr. 
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not administer d^5ai±inental, district, or school-wide tests. 
Differences between high and Icwer scorers did emerge, however, 
vAien- teachers were asked to evaluate the effects of state and 
Advanced Placement exams on their instructional efforts, of the 5 
teachers havlj^ had experiense with AP courses (i.e., 4 lower 
scorers and 1 hi^ scorer) , the 4 lower scorers feel that AP tests 
"sv^^xart" a tliiridng emphasis vMle the one hi^ scorer believes 
such tests "inhibit" his classrocm efforts to prcroote students' 
thirfdng. liJcewise, at the one school administering a state test, 
the two hic^ scorers think the test inhibits a thinking eitjiiasis 
vd)ile their Icwer scoring colleague sees the same test as 
si^^xartive. As will be explained later in this chapter, 
differences here between hic^ and Icwer scorers my be attributed 
to differences in their instructional goals and their resolution of 
the depth vs breadth dilemma. 

To summarize findings on organizational barriers, the primary 
ooncem of teachers, v^aether identifyij>g central barriers or 
suggesting ways to improve their instructional efforts throu^ a 
hypothetical "wish list", is to have their student load reduced. 
Aside frcm the barrier of content coverage pressure cited by hi^ 
scorers (an issue that will be addressed else?*fire), teachers' 
pero^xtions of prcminent barriers and ways to organizationally 

facilitate students' thinking are quite varied ^both within and 

between the two grcxapB. Seme data suggests however that hi^ and 
lower scorers may differ in their perc^jtions of the effects state 
and AP exams have on the pnanotion of students' thinking. 

i x^m VS mnAum of cxxrmm cnvERflCE ; 

Teachers were asked a variety of questions to elicit their 
views on the issue of depth vs breadth of content coverage. As a 
vAiole, responses to these questions reveal more difference than 
similarity in the thinking of the two grcnqps. 

Both groi^js of teachers unaninicusly responded, "yes", vAien 
asked if they have a conflict between depth and breadth of content 
coverage. In addition, roost of the hi^ scorers (4) and a majority 
of lower scorers (3) view "coverage pressure" (i.e., greater 
breadth) as a "fairly negative" or "extremely negative" influence 
on the pranotion of students' thinking. However, v*ien asked an 
cpen-ended question in vdiich they were to esqplain the nature of 
this ccaiflict, differences emerged between the gjoiips. Tteachers' 
responses were categorized as indicating either an external (i.e., 
other than self) or internal (i.e., self) source of breadth of 
coverage pressure, with a few teachers indicatii^ both (see Table 
9) . All of the hi^ scorers (5) cited external sources of coverage 
pressure ccaipared to only a few among la/er scorers (2). For 
exanple, hi^ scorers cited the following external sources: state 
exam, state or district irtposed curriculvrai guidelines, AP course 
outlines and exams, students' lack of knowledge, the department 
head, and colleagues. Ihe few Icwer scorers vdio did cite external 
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scuroes loenticaied the AP program and district crurriculum guides. 
Differences between hi^ ax)d lower scorers were iDiniinal with 
respect to an internal- self source of coverage pressure, as two 
hi^ scorers and oie Icwer scorer identify self as the source of 
their breadth pressure. 

Results ccffisistent with the above were obtained in a later 
school visit vflien we asked teachers to raiik in order of inportance 
the following poten t ial sources of coverage pressure: myself; 
d^>artinent colleagues; deparbnent liead; school administratis; 
district tests or guidelines; state tests or guidelines; and, other 
(see Table 10) . Alncst all of the hi^ scorers (4) ccnpared to 
only a few Icwer scorers (2) identified sources other than 
themselves (i.e., external sources) as the primary caiise for their 
coverage pressure. Conversely, most lower scorers (3) cited 
themselves as the primary cause or source for their coverage 
pressure caipared to cmly one hi^ scorer. 

To review, all of the teachers say they ejqjerienoe a content 
coverage conflict. High scorers esqperienoe the conflict as 
primarily a situation in \Aiich external sources inpose coverage 
de m a n ds that exceed the pace desired by the teacher, whereas lower 
scorers tend to inpose coverage demand on themselves. Hiis doesn't 
mean that hi^ scorers do not pressure themselves toward greater 
coverage, only that for hi^ scorers external sources seem to 
demand rsore coverage than the teachers themselves deem ex p ropriate, 
lew scorers, however, are either in agreement with the coverage 
agenda of external scutces or perhaps they desire even greater 
coverage. Bie findings that follcw si^port these initial 
generalizations . 

Wfe xso asked teachers to identic the three factors that most 
inhibit the prctDtion of students' thinking. Respondents were told 
to consider all facets of their working situation, including school 
and classrocm organization, curriculm and instruction jg^^ieg and 
materials, students, administration, etc. iOmost all of the hi^ 
scorers (4) idantified coverage pressure (e.g. , state exams, 
district guidelines, state guidelines) as ca^ of the three most 
detrimental factors to their efforts to prccote students' thinkit^. 
One lii^ scorer went so far as to cite 2 forms of breadth pressure 
among his 3 most detrimental influences, (i.e., a state exam, and 
the district's course guidelines) . Only a few of the Icf^ scorers 
(2) citad coverage pressure, with one of these tea^diers later 
admitting he wasn't sure vA-^at he'd do if given the time to pursue 
topics in greater dqoth! All sources of coverage pressure 
Ixaentified by hi^ and Icwer scorers were external in origin. We 
also placed teachers in a hypothetical situation in vAiidh they were 
granted three wishes to help them prcmote students' hi^ier order 
thinking. Diey were to view the question as broad ranging, and^ 
again, the results were consistent with the above findiiigs. Most 
of the hi^ scorers (3) cccpared to oaLy one lower scorer requested 
less content coverage as one of the three most important sdiool 
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features needing modification to pronote students' thinkir^ more 

Another interesting difference between the two grocps can be 
seen in their attitudes toward ttie breadth oriented Advanced 
Placement (AP) courses, AP history courses held stuteirts 
accountable for understanding a vast sweep of history for 
successful ejom ocupletion, though portions of the i jv\L are of an 
essay format \Mch challenges students' rcjasoning abilities. Of 
the 5 teachers having had ejqperience vdth AP courses, 4 were lower 
scorers and 1 a hi^ scorer. All of the tt^chers cited immense 
coverage pressure in their respective AP course, yet only the hi^ 
scorer felt that the AP exam "irihibitec''* the proiaotian of students' 
thinking. All four of the lower scorers on the other hand assessed 
the AP course as "supporting" the prcrootion of students' thinkirg. 
Similar findings emerged at the one school in the saraole that 
participated in state testing. Like the AP format, this particular 
social studies state exam (and the curriculum guideline that 
'drives' it) requires extensive content coverage for successful 
ccjtpletion. Ohe two hi^ scorers claimed the state exam inhibits a 
thinking en?*asis while their lower scoring colleague felt the . jte 
exam is si:5portive. 

We also asked teachers directly to explain the natixce of their 
coverage ccaiflict and hew they attenpt to resolve it. Responses 
si?5X3rt the above findings that hi^ scorers are nor.e 3.ikely than 
Icwer scorers to perceive coverage pressure as externally inposed 
and deleterious to their teaching efforts. In addition, teachers' 
responses hera indicate that hi^ scorers generally attenpt to 
resolve the conflict in the direction of less coverage ocnpared Co 
lower scorers by ozceSaHy selectirg and prioritizirg topics, 
ideas, and issues. 

Turning first to thsi "•••.'..cionses of hi^ scorers, Harold 
ackncwledges the coverage pre'ikire he eJ5>9riences fran state 
guidelines and the state exam. He states: 

...I would like to have the freedan to be 
flexible, you knew, like today's lesson on the 
Immigrants. It really worked. Kids were 
talking about vfliat the ejqperience would i.^e like 
and they would do in those circunstances. 
Some were saying that they would go hcsie. 
Others felt that they would stick it out. I 
would like to stay with a topic llJce that. I 
could think of numerous issues I would like tn 
pursue about this topic. But I can't.... You 
are obligated to finish the curriculum. You 
also feel pressure from the chairman. 

Nonetheless, Harold goes on to say: 
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I c3o not preocc^ iryself with finishing the 
curricuiiim. Instead^ I atteirpt to teach 
v*atever I teach well aixi select classrocm 
topics and itaterials very carefully...! don't 
enpphasize content coverage. It's Ixadicrous to 
atteafrt: to cov^er 100 years of history in a 
iQonth or two. I focus on cano^)ts and ideas. 
Ihe problem with roost sdiool courses is that 
they are survey courses that are hccogenized. 

Hansen, another hi^ scorer, argues that teacheis* atteirpts at 
broad coverage are docraed to failure since ".,.th-e anomt of 
knowledge in the world has doubled in the last 8 years. So I see 
ny jck» as enabling students to be 'good thinkers. He points out 
that the 6sgih vs breadth conflict is really not the issue as: 

"...you could survey all of American History 
in a boring, non-critical, non-thinking way or 
ycu could focus for the ^Aiole 18 week semester 
on the causes of WWII in a boring, non-thinking 
nan-critical way. Just going into d^jth 
doesn't necessarily mean you're teaching kids 
to think." 

For Hanson the «intral issue is w^^ther or not one eitfiiasizes 
critical thinking: 

"Your ccscimitroent to praroting thinking ^^vems 
your choice of curriculum. . .When I plan 
curricula I side on the depth side of the 
conflict by leaving a lot of things out. I 
select material which is representative of 
particular conc^rtzs and gei^ralizations I wish 
to develop. I hope that the thinking skills 
developed with these representative selections 
will be transferred to other examples... I 
think you can teaoh almost anything in a 
critical way, in a way that gets kids thinking 
- but it takes time." 

Turning to the responses of lower scorers, Lisa, also 
e>5>eriences breadth of coverage pressure, pa2±iailarly in her AP 
course: 

Ihere is a ccarprehensive exam given the second 
week in May covering all the material ftxan the 
1600 •s to the p:.-esent. Ihis puts tremendous 
pressure on ny classes to move through the 
material quickly. In our discussion of slavery 
we have had little ti2»5 to e)^lore the long 
range implications of this e35)erience...Ihere 
is sinply not enough time. 



in - 18 



Yet, \)tven. asked hew she would attenpt to resolve the cavera e 
conflict if the AP course did not "iitSDose" a curriculum of breadth, 
Lisa, unlike hi^ scorfers, still sid^ toward breadth: 

If I had to make a choice I would choose 
coverage because it is perhaps their only 
esfflerienoe and prcbably a oonclxsding ea^jeriencs 
with history. I really feel an cbligation to 
at least eicpose them to scroe of the pressing 
issues of cair time. Not to get to the 1950's 
and 1960's, the Cold War... is 
uncoiscionable. ..I think it is more inportant 
that they get e)5X3sure ani that has to 
sacrifice dqpth. ..I want them to get as mach 
ejqposure as I can, you know, shove it down 
their throats. 

Lisa's ccmnents reveal an internal source of coverage pressure 
of at least equal magnitude to the external pressure iitpcsed by the 
AP curriaiiim guidelines. pjsgardiess of ""the covrerage maMates 
dictated by the AP course, Lisa's resolution of tlie breadth vs 
dqpth dilemma indicates an allegiance to es^xsing students to 
issues rcther than to explorirg issues with students. 

LLcyd, as mentioned previously, looks 'to the administcatiax 
for instructional guidance. Becai:ise his scope and sequence 
guidelines and 'skill' objectives are provided for him and 
acc^jted, he had difficulty rt-lating to our ccn<erage dilemma 
prdDes: 

I sort of have an idea vtot you're talking 
about but I think that issue of dqpth and 
breadth doesn't oane \3p for me. Ifcwever, he 
does reveal his breadth orientation (and 
presumably the district's) vdien he states; "I 
can't think of or^ area that I've ta^i^t vbsre 
I'd want to go that much in depth. 

Finally, we asked teachers to assess their coverage 
orientation with respect to particular lessons. Again differences 
emerge between the two grc(i:?)s, differences that are consistent with 
the above findings. Teachers were asked. If their approach to the 
tcpic of the observed lesson is best characterized as oriented 
tcwcird d^jth or breadth of coverage. All of the high scorers felt 
their respective lesson indicated a depth earphasis viiereas all of 
the lower scorers viewed their lessons as breadth oriented. Yet 
vdien asked if their approach enhanced, inhibited, or had no effect 
on the quality of students' thijiking, all of the hi^ scorers and 
all but one of the lower scorers felt their ajproacdi enhanced 
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students* thirikingl^^ 

A careful interpretation of the above findings suggests that 
Icwer scorers, unlike hi^ scorers, are cau^t in a contradiction. 
That is, Icwer scorers make the general statement that breadth of 
coverage is detriinental to thiriking, yet at the same time: a) claim 
that specific breadth-oriented lessons erihai^ students' thinking, 
and b) inpose coverage pressure on themselves equal to or greater 
than the coverage demands articulated by the dg>artmsnt or 
district. To understand lower scorers' apparent contradiction here 
requires a return to their instnactional goals. Lcwer scorers tend 
to Hnjliasize various forms of COTtent aoquisiti^ as their primary 
instructiona]. goal \diereas high scorers view content more as a 
•vehicle* to prcaraoting thinking. For lower scorers, thinking is 
incoi^rated into their priitary mission of teaching content, 
xAiereas for hi^ scorers thinking is the central focus, with 
content understanding as a necessary ccnponent. Siix^e lower 
scorers place greater eitftesis on cont^^ acquisition it is not 
surprising that cccpared to hic^ scorers th^ tend more toward 
'breadth* of coverage. This would eoplciin lower scorers' tendency 
to self-isposa coverage dsmaixSs equal to or greats than the 
coverage demands of their social studies departmsnt or school 
district. (Biis may also e)5)lain their positive assessnasnt of 
breadth-oriented Advanced P\acenent courses) . 

It is a bit more difficult to e&5)lain lower scorers' belief 
that in genr^ral breadth of coverage is detrimental to a thinking 
enfiiasis vftxen at the same time they feel that their own specific 
lessons oriented tofard breadth prcmote thinking, it is possible, 
however, for lower scorers to be consistent here. Uiey could 
believe that their breadth-oriented lessons prcmote thirfcing but 
not as much as a d^th-oriented lesson. Bieir 'breadth lesson' 
would then be consistent with tiieir overriding primary goal to 
expose students to many facts and ideas in the social studies. 
ISifortunately, a follcw-i?> probe was not attecpfced with respect to 
this issue. Ifcwever, the dissatisfaction seme of the lower scorers 
expressed about their breadth-oriented lesson during a 
post-cbservatiOT interview, and the almost apologetic tenor of its 
ej^pression, suggests that lower scorers have not resolved their 
coverage di l e mm a in the finely reasoned ronner constructed above. 
Rather lower scorers seem to vacillate between teacihing for content 



■^■^-IXie to illness or sncw days 4 of 5 hi^ scorers and 3 of 5 
lcwer scorers were asJoed coverage questions relating to a specific 
l^son observation. Hds is unfortunate as the 2 lower scorers not 
asked were most candid about tl^ir breadth orientation, making it 
very liJcely that 5 of 5 lower scorers would have described their 
lessen as characterized by breadth. Whether these two teachers 
woold see their respective lesson as enhancing or irtnibiting the 
qiiality of students' thinkiryy is less clear. 
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coverage that is both self and e>ctemally inposed, and a desire to 
engage students in thinking and nore substantial anal3^is of topics 
\Mdi reqoires greater depth of coverage. Further research is 
clearly necessar y on this natter. 
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KKMiixiL'icMS car giiJiaans ; 

Hi^ and Icwer scoring teachers unanimously agree that at 
times students resist school work that demands thinking. Ihe types 
of thinking taslcs teachers identify as inost jErequently resisted by 
the less recejrtive students are extreraely varied and difficult to 
classi:^ in ways that mi^t distinguish between hi^ and lower 
scorers. Of the 20 or njore thinking tasks identified hy teachers^ 
in only two instances did two teachers froa either groip identify 
the sane task as most firaquently resisted hj less receptive 
students. Ihe ccnclusion to be ^3rawn here is that teachers 
perceive students as resisting a wide variety of tasks requiring 
thinking.^ 

Botli grt3i55s of teachers unaniiaously agree that scane students 
are more reogrtdve than others to assignments, activities, and 
tasks requiring thinking. Using a s'-^t of structured questionnaire 
items teachers were asked to refl*3ct not only on factors leading 
scane students to perform "above average" on thinking tasks, but 
also to refill* csi factors that influence ^Hoelcw average" 
performance. We were curious to see if teachers enjiiasized 
different factors with above and below average 'student-thinkers*. 
Teachers were to rate on a 5-point scale (1= no influence to 5= 
major influence) the degree to \fliich they believe each of 12 
factors play a role in the performance of above and bnlow average 
•student-thinkers. 13 mean score was computed for both groips 
of teadiers on each of the 12 factors. The 3 factors receiving 
greatest enjiiasis (i.e., the hi^iest mean scores) by each groip 
(see Tables U and 12) were cccpared and wi3.1 be discussed below. 

Die data suggests that hi^ and lower scoring teacters agree 
v¥>on the most iitportant factors for student performance on thinking 
tasks. Both grocps enjiiasize the same 3 factors, .(out of a possible 
12) attempting to e35)lain the performance of above average 

student-tiiinkerri, and 2 of 3 factors with respect to below average 



^However, teach^srs* responses here do surest that the tasks 
hi^ scort: rs pose to less rec^>tive students are more readily 
identifiable as tasks raguiring thinking than are the tasks posed 
by lower scorers. Ihis will be discussed later in this section. 



^^The 12 factors are peer group pressure, family 
backgroxind/situation, students » enplqyment responsj±)ilities, 
students' ejqperiences in prior social studies courses, students* 
rajport with teacher, students • irherent cognitive capacity, 
stidents* learned thinking skills/abilities, students* present 
kiKJWledge, students' writii^ abilities., students' level of 
self-confidence and self-esteem, students* motivation, ani any 
other factor not mentior^ above. 
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student--thlrikers. Bie three factors both groi^ps ar^hasize with 
above average student-thinkers eure " leeamed thinking 
skills/abilities"^ "cx5gniti\^ cs^city", and 'taotivaticn".^^ OSie 
two factors both groups of teachers ertphasize ^.th below ewerage 
student-thinkers are (a lack of) ''raotivation" and (lack of) 
"self-conf idencje/estesm" . 

Second/ hi^ and Icwer scoring teachers do not eniphasize •th'^ 
importance of "cognitive capacity" with below average 
student-thinkers tut do so with respect to above average students. 
Both groups apparently feel that factors other than cognitive 
capacity best ejqolain the inability of low perfontdi>g students to 
ejdiibit at least avei-^ge if not above average parfonnance on 
thinking tasks. 

Ihird^ "inotivatj.on" is the only factor to be emnhasized by 
both grtxpQ of teachers with respect to both types of students. 
Teachers may intuitively knew \*at educational researchers are now 
only beginning to suggest (Dweck/ 1986) ; that is, the develcpasnt 
of good thinkers most furriamentally involves changii^g students' 
gene23l attitudes aivS dispositions ratJier than csily ejsHrdii^ 
students* knowledge stractures/ iuproviTig cognitive processes^ or 
developing specific thinking skills. 

Fourth/ the relatively hi^ irean score across all 12 factors 
(i.e./ 3.81 or 3.83) suggests ">iat both grji^ps of teadiers believe 
a variety of factors play a role in students' performance on 
thinking tasks. Teacbei^ and not jus** researchers realize the 
ccxiplexity of thje instructional/learning/ thinking process. 

Possibly more inpoitant than teacdier perceptions of factors 
that inflvience student performance on thinking tasks is the 
teacher's sense of inflxience on these factors. We suspect that 
the more a teacher believes he/she can infliience factors associated 
with thinking, the hi^ier will be the teacher's e^qpectations for 
student performance. 

Teachers v^re asked to rate on a 5-point scale ( no 
influence to 5= major influence) the degree to vMdi they believe 
they can influence in cne year 7 factors contributing to students' 
performance on thinking tasks. ^ A mean score was ccjipited for 



^^IXie to a three-^y tie in mean score on the third most 
iirportant factor cited by hi^ scorers, 5 rather than 3 factors are 
listed. Ihe two other factors enphasized by hi^ scorers are 
"self- confidence/esteem" and "teachers-student rapport." 

^^Ihe 7 factors are: students' rapport with teacher/ students' 
learned thinking skills/abilities/ students' present knowledge/ 
students' writii^ abilities/ students' level of self-confidence and 
self-esteem/ students' motivation/ and 'Mother'. 
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both the hi^ and Icwer scoring grcx^B on each of the 7 factors. 
Ihe 3 factors hi^i^ted by each gnx^) as being most amenable to 
positive inflxieiK^e (see Tables 13 and 14) v?ere occipared and will be 
discussed below. Ihe data suggests that hi^ and lower scoring 
teachers hold very similair views regarding the factors they 
believe they can most inflxience with students in the course of a 
year* Out of 7 possible performance factors both groins believe 
they can most influerr^e the same 3 factors with respect to above 
average student-thinkers, and 2 of 3 factors with respect to below 
average student-thinkers. Ihe three factors both groips eicphasize 
with above acverege student-thinkers are "students' knowledge base"/ 
"teadier-student rapport"/ and "students' writing iddlls".^^ nie 
two factors both groi^ of teachers errsphasize with below average 
stiident-thinkers are "students' thinking skills/abilities" and 
"teachers-student rapport". Only one factor was consistently ranked 
as hi^y amei^able to teacher influence among hi^ and lower 
scores, regardless of student type. Ohis was "teachers-student 
rapport." 

We found that lower scorers believe th^ can influence the 
performaiw. of above average stident-thinkers isxyxo. than hi^ 
scorers/ for the mean score of lower' scorers across 7 factors is 
3.69 vs 3.20 for hi^ scorers. But this oanclojsion is siispect/ 
because a number of hi^ scorers during interviews mentioned that 
they felt little iTeed to influence or iitprove scaae factors with 
above average student-thinkers; these students were already 
outstanding in such areas as motivation and teadher-student 
rapport. 

In contrast/ high scorers believe they can influence t:?e 
performance of below average student-thinkers more than lower 
scorers. Pour pieces of evidence support this conclusion. First/ 
the mean score for hi^ scorers across all 7 performance factors is 
3.61 vs 3.08 for \:Mer scorers, a difference of over .5 per factor. 
Second/ in a direct probe of tiiis issue/ teachers \^re asked 
v*iether or not it is more difficult to enjiiasize thinking with low 
achieving students. 17 Most hi^ scorers (4) / but only a few lower 
scorers (2) / said it was not more difficult to emphasize thinking 
with lew achieving students. For exsnple/ Hilary/ a hi^ scorer/ 
^ays she "sta^ with a student" vdio responds "I don't know" to an 
opinion/ analysis/ or evaluation question. "I let them know at* 



l%ie to a two-way tie in man score for the third factor 
among hi^ scororS/ 4 rather than 3 factors are listed. Ihe one 
other factor enphasized by hi^ socjrers is ••tiiihking ddLl^abiLitx2s." 

^"^An assunption is made here that ^*ien teachers think about 
their lew achieving students and their belcw average thinkers they 
have/ essentially/ the same population of students in mind. 
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the beginning of the year that an •! don't know* answer is 
ixnacx^^jtable, th?at I will not go away until th^ get it ri^t," 
Hilary feels that "prior e55)erience, lack of confidence, and 
inental laziness" have ccxribir^ to loake "I don^t know" a safe 
response, one that •'•will make the teacher seek a response 
elsej^iere*" However, "after a few successes they want to be 
called on and try agedn." 

A third strand of evidence comes frm an interview in which 
teachers were asked to ejqplain the lacik of involvement on the part 
of sacDe sti^ierrts in a lesson recently observed by the researxiier. 
All of the lower scorii^g teachers (5) blamed students themselves 
for lack of lesson involvement \Anle only a few hi^i scoring 
teachers (2) did. lower scorers suggested that students don't like 
to talk, don't feel like working, aren't interested, haven't done 
the reading, are lazy, fear ridicule, cannot articulate their 
thou^ts, etc. Bie few hi^ scorers vflio blained students loentioned 
the following three student deficiencies: some have personal 
problems, are afraid to risk their ideas publicly, and, don't have 
a chJB as to hew to make a predicticryhypothesis. Recall that of 
the olBBsror^ dDserved, most lov^ scorers (3) had classes 
ccfflDposed primarily of hi^ achieving students, \ghereas most hi^ 
scorers (4) had either a diverse range of achievers or 
predoaoainantly lew achievers. Yet one finds the lower scorers 
blamijng students for lack of lesson involvement, not the hi^ 
scorers. 

A fourth piece of evidence suggesting that hic^ scorers feel 
more influential with below average student-thinkers is that a 
majority of high scorers (3) but riO lower scorers plaoed partial 
blame on themselves for disaigagement among sam students during a 
specific lesson. Hi^ scorers ijvsriminated themselves in the 
following ways: hadn't iearr^ed the names of all the students and 
therefore had to inpersonally point his finger to get responses, or 
avoided calling on soane students; had kids continue to giv^. tb2ir 
opinions even though responses were redundant vMok led saa^ 
students to tune-out; calling on kids row by row rather than 
randomly vMcdi led sme to tune oait until it was their turn to be 
called on; and, needed to spend more time with particular students 
on their responses. Self-attributions hi^ scorer here 
suggests greater locus of control on the part of ttiese teachers; 
the responsibility for prcanoting thinking seems to rest squarely 
with the teacher, not with the student. 

To summarize this section on teachers' percqptions of 
students, mucii agreement exists between hi^ and lower scoring 
teachers. Both groi5)s unanimously agree that students at times 



■*-°An assunption is being made here that the students vdio were 
not engaged in the lesson are also, in large part, the students who 
perform below average on thinking tasks. 
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resist thinking tasks, and that sane stuJents are nore receptive 
than others to tasks requiring thinking. Both grot5>s euiphasize the 
same factors (3 of 3) as laost ijiportant for the perforroanoe of 
above average student-thinkers, and the saine factors (2 of 3) vath 
respect to belcw average student-thirOoers (see Table 15) • Both 
grtx^js are also in agreeinent regarding the thinking performance 
factore they feel th^ can nost influence with students, Hi^ and 
lower scorJiig teadiers believe th^ can nost influence the saine 
factors (3 of 3) with respect to above average student-thinkers, 
and the saiae factors (2 of 3) with respect to belo&/ average 
student-thinkers (see Table 16) . 

Bie two groips seem to part ways, however, in the degree to 
vMch they believe they can influence the performance levels of 
belcw-average student-thinkers. Hi^ scorers ccarpared to lower 
scorers are more likely to say that it is not more difficult to 
eirphasize thinking with Iw achieving students, and blame 
themselves rather than students for lack of student lesson 
involvearsent. 

OCWCEPi'IOHS OF gllNKIMS ; 

Since prior research shows that many of the idaas guiding 
teadiers' practices are implicitly understood and are therefore 
difficult for teacihere to articulate (dark & Peterson, 1987), we 
used several questions to elicit information on teachers' 
conoqptions of thinking, Cie two most direct questions were: 

-Do you have a conception of thinking that 
guides your teaching? If so, summarize its 
main aspects. 

-Consider your best thinkers, what 
distinguishes them from your other students? 

Ihe first question asks teachere directly to e)5)lain their 
conception. Ihe second question, an indirect probe, assunes that a 
teaoher^s conception of the good thinker is closely related to 
his/her conception of thinkirg. Tfeaoher responses to several other 
questionnaire items that further illuminate teachere' conceptions 
of thinking were also included in the analysis. 

Ihe feamework developed to help analyze teachers' conceptions 
of thinking dram vpon major distinctions made in scroe of the 
scholarly literature on thinking, specifically thinking as 
dispositions, as content mastery, or as skills. With regard to 
di^x)sitions, Schrag (1987) has argued that thinking is closely 
linked to the character traits of reflectiveness and mental 
flexibility. Other dispositions frequently associated with 
thou^fulness include: curiosity to question and' e)$)lore"; an 
insistence that bairns be siqpported by reasons (and that the 
reasons themselvcss be scrutinized); confidence in one's thinking; 
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motivation; and, a vdllingness to view an issue or prx±>lem frcan 
different perspectives. HSre are a few exanple-excerpts of 
dispositional language talcen fron teachers' concqptions of 
thinking: 

-A good thinker isn't afraid if jsomeone 
challenges a position. A good thinker is 
willing to take a look at scneone else's 
hypothesis or theory even if it's 180 degrees 
apart frcm his cwn, rather chan a dogmatic 
kncwer. 

-I want to develc^ in studeits an attitude that 
nothing is finally closed... If that's skeptic' 1 
I don't think it's skeptical in a negative 
sense. 

Hie second ccnpanent or perspective on thinking hi^i^ts the 
inportance of knowledge of subject matt/a: understanding for 
effective thinking, as suggested, for exanple, in recent research 
on ejmert/ncvice prc&lsm-solving and \'ariaas information processing 
models of thinking and learning. According to this perspective, 
one cannot think in a ccaifcent vacuum, that is, one must have 
scanething to think about. Effective thinking about a prcblem or 
task occurs only \*en one possesses sophisticated understanding of 
the content to v4:lch one's thinking it directed, be it elevator 
repair or open-h- surgery. Teachers also emphasized content 
tmderstanding; for exanple: 

-You build off a knowledge base vMch is 
essential. 

-You can only think vdien the^^ have enough info 
to get a detailed picture of an event. 

Finally, the skills perspective defines thinking in terms that 
usually transcend specific subject matter. A good thinker, for 
example, can detect bias, identify a prcblem, muster evidence, and 
analyze or evaluate a boc^ of material, etc. Skills can be 
construed as domain- specific, for exaitple, legal reasoning in 
law, or solving proofs in geometry. In shcrt, the skills view 
states that students must possess a variety of skills, techniques, 
strategies, and heuristics vdien approaching a prcSslem or task. 
Teachers also used a skills language of thiiudng; 

-^Ihihking is determining validity based on 
reasoning, intuition, and emotional skills. 

-There are five process skills; recall, 
organization, analysis, synthesis and 
evaluation. 
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Simply counting teachers' responses to various questionnaire 
items revealed little difference in the frequency with vMch hi^ 
and Icwer scorers use dispositional, skill, or content language to 
describe their conceptions of thinking. For exanple, when teachers 
were asked to describe their ccnc^jtion of thiriking, hi^ scorers 
primarily used skill (5) and dispositional (3) language (tjee TeiblQ 
17). lew scorers » responses are very similar; that is, skills (4) 
and dispositional (5) language predcminate, vdth some teachers also 
referrj^ig to content (3). Similarly, vAien teachers were asked to 
e:q)ledn hew cJsserved lessons (2-3 lesscsTS per teacher) prcooted 
students' thinking (see Table 18) , the responses of high and lower 
scorers again revealed little difference. Based on a total of 13 
lessons from each grtxp of teachers, all high and lower scorers (5) 
used content language, all hi^ (5) and nost lower scorers (4) used 
skills language, vMle only one teacher frcm either grcqp used 
dispositional language. ■'■^ 

Ursxn^ both groiqps respond in like fashion to the two 
different prcbes into teachers' conc^jtions of thinking, notice 
that tliere is a shift in language usage. The greater emphasis 
placed on content language wiien discussljig lessons is prcbably due 
to the fact that teachers are not talking here in general terms, 
but rather about specific thiriking tasks with respect to specific 
content. Ihe lack of dispositional language is not surprising, 
one probably wouldn't expect a teacher to emphasize the 
dsvelcpttent of thou^tful dispf>sitions in a time frame of one 
lesson. Whp.tever the actual rea«)n(s) for the shift in emphasis, 
one can conclude that hi^ and Icwer scorers use dispositional, 
skill, and content language in rou^y equal measure vftien talking 
about thinking generally or with respect to specific lessons. 

Significant differences* begin to emerge between the two 
grotqps, however, when teachers' concqpti.ans are viewed more 
qualitatively and in their entirety; that is, hi^ scorers' 
exceptions of thinking tend to be lengthier, nore detailed, and 
more elaborate than the conc^ions of lower scorers. The 
thinking concqption of one teacher frcm each gra$) will be 
described to hi^i^t these differences. Ihe hi^ and lower 
scoring tei.cher to be deBcribed here possess neither the most nor 
least developed concqrtiion of thinking in their respective groip. 

Hilary, a hi^ scorer, describes her conception primarily in 
skill and dispositional language. she mentioned skills that 
include the ability to "analyze", "focus on an argument aM 
support it", "determine cause ana effect", "induction", 
"evaluating, based ttpon si?5»rtable evidence", "determining point 



^^Skill language vas used by hi^ scorers on 10 of 13 lessons 
and by lower sec :ers on 7 of 13 lessons. Content language was used 
by hi^ scorers on all 13 lessons and by lower scorers on 11 of 13 
lessons. 
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of view and identifying its effects", "see relationships", and 
"draw conclvisions." She enjihasizes that "the teaciier must give 
students the cRportunity to develop these abilities." 

In addition, Hilary subscribes to the Grandville Hi^ Sdhool 
skills model of thinking because she's "in the d^)a3±3nent and it's 
so central to the program." liiis model ccnibines the work of 
Bloom, and Beyer's skills approach to thiridi^. Hilary is 
well-versed in the specifics of the model, as she quickly 
identified the ccnponent thinking skills (i.e., recall, 
organization, arcilysis, synthesis, evaluation, and divergent 
thinking) and then discussed their hierarchical nature; 
"The 'higher' skills entail the skills belw... organization 
involves recall, analysis also involves organization and recall, 
synthesis involves analysis... Divergent thiiiidi^ is at the top and 
is the hardest for students to do." 

Biou^ Hilary is versed in skills language, she also 
enfJiasizes iitportant elements of thinking with dispositional 
language. In fact, one of the most iitgportant attributes of the 
good thinker for Hilary is "intellectual curiosity." she 
enphasizes that "students vto perform best on thinking tasks often 
have an intellectual curiosity. . .not necessarily equated with 
hi^iest 'inherent cognitive capacity'." Hilary encourages 
students to ask questions, and ultimately wants them "to see that 
questicai asking is a sign of intelligence not ignorance; to get 
them to knew vAiat they don't knew and want to find out." She finds 
the attribute of intellectual curiosity to exist not only at the 
individual level, but also at the classroom level. It is this 
'type' of student, and class, that she most enjoys. 

A3 discussed earlier with respect to Hilary's gcals, she 
places great eirphasis on developing students' appreciation of and 
willlngnesc to ej^perience altzemative perspectives. Above the 
blacikboard on the front wall is a poster statii^ the following: 
"Where you stand depends on vdiere you sit." OMs attitude, or 
general understanding or belief in the iirportance of perspective 
is, according 1>o Hilary, "essential for students' thinkii^." with 
great elaboration she describes lessons she uses to bring this 
point or 'viewpoint' across to sUdents. In fact, 'perspective 
taking' is the topic of Hilary's very first lesson each semester 
and provides the central theme for her courses. 

Consistent with her eitftesis on perspective taking and 
intellectual curiosity, Hilary tries always to keep the following 
in mind \ghen leading a discussion: "Never ask a qu^td-on v^iere you 
already knew v*at you want for an answer." Another di^xssition 
she finds iirportant for thinkii^ is a "willii^ess to take rrisks". 

When discussing how specific lessons challenge students to 
think Hi]/ary states tiiat she much prefers to talk about her 
efforts with respect to th? actual content of the lesson. 
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CXiriously, Hilary clcdms she lacks "the cognitive scientist's 
language to describe thinking^* (except for the dq)arbiient»s 
6--skill thinking JKx2el), and cSoesn't consciously apply it i^Sien 
lesson planning or engaged in classrocD instruction. Instead, she 
"intuitively knows vAiat questions are »hardS 'harder', and 
'hardest'." Hcwever, her cjiite elaborate cancesptian of thinkii^, 
cccbined with referenote to specific classroaa behaviors she 
enplcys to facilitate students' thinking, may belie the above 
connents minimizing the litportance of a thinking concqation for 
her instructional success. 

Lloyd, like most of his fellow lower scorers, provides a 
fairly brief conc^rtlon of thinking, one that irrplies concem for 
dispositicais aixi skills. licyd hic^i^ts several skills: "ability 
to connect seemingly unrelated ideas", "ability to octnoiunicate", 
"categorize or grcKsp things", "use application skills", and 
"interpretation, categorizing, sequencing. • .analysis. . .and 
evaluate the facts". LLoyd mentions that he has read Norris 
Sanders', Classroom Questions , and finds that "it is useful 
because ycu can thumb throu^ the book for his hierarchy and see 
seme applications of the qusstionii^ techniques." Ihouc^ Lloyd 
did not summarize Sander's hierarchy, it is reasonable to assume 
he kncws, because many of the skills he mentions are also used by 
Sanders (i.e., interpretation, application analysis, eval\iation) . 
Lloyd's concem for dispositicms emerges vSien he defines 
intelligenoe as "curiosity and eagerness and ability to connect 
different ideas." 

To briefly summarize this section on teachers' conceptions of 
thinking, hi^ and Icwer scorers tend to use content, skill, and 
dispositicml lar^uage in equal measure vAien discussing their 
cono^^tion of thinking geivarally or with respect to specific 
lesson efforts. Inportant differences begin to emerge however viien 
teachers' oano^rtdons of thinking are viewed more qualitatively. 
Biat is, most hi^ scorers octpared to only one lower scorer 
offered a COTception of thinking that was fairly detailed, 
lei^thy, and elaborate. 20 

OOKdDSICM : 

OMs exploratory stud/ of 5 areas of teadiers- thouc^it (i.e., 
goals, barriers, d^th vs breadth, perc^jtions of students, and 
conceptions of thinldi^) founi both differences and similarities in 
the thinking of teachers scorii^ hic^ or low in their protntion of 



-^"Spaoe limitations prevent the presentation of additional 
ejcmples to help substantiate this point. However, with respect to 
length, summaries of hic^ scorers' conceptions of thinking acveraged 
47 lines of text viiereas lower scorers' summaries averaged 19 
lines. See Qnosko (1988) for cojiplete sumnaries of each teacher's 
caic^ion of thinking. 
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thouc^itful classroott disccwrse. Individual questionnaire iterns 
revealed cxirplete agreenent within the or lower 

scoring ^ dranatic differences between them. But vAien 

results I:ed questions were conbined, soroa consistent 

patterns esmirgt'a that indicate areas of siinilarity and difference 
between grctJps. Both hi^ and lower scorers hold similar views on 
barriers to pctrooting students' thinking, specifically large class 
size and tot-il student load. Uhfcrtunately, neither graap 
su^ested straii-^ies to minimize the effect of these barriers. 
Instead, we fcxxvd a pervading pessimism about change, especially 
with respect to i^rtxdent load. 

Many simil?irities in the thinkirsr of the two groqps also 
emerged with respect to stutaents, especially in the factors 
perceived to most injortant for succsessful performance on 
thinking tasks, and the factors that teachers believe they can most 
influence with sstudents. However, various strands of evidence 
suggest that hic^ scorers cotpared to their lower scoring 
ccunterparts feel more ef f icacicws when it ceres to prt^notir^ the 
thinking of belcw-average student-thinkers. 

Iroportant differences seem to exist between the two groups 
with respect to their instructional goals and their views on the 
issue of depth vs breadth of content ccverage. Lower scorers tend 
to enjiiasize content acquisition as their primary instructional 
goal vAiereas hi^ scorers view content more as a 'vehicle* to 
promoting thinking. Stated another way, thinkir^ is incorpore Jd 
into lower scorers' content mission, vdiereas hi^ scorers place 
thinking as the central focus with content understandii^ a valued 
outccane. Ixjwer scorers prefer to expose stidents to ideas and 
issues, whereas hi^ scorers prefer to explore ideas and issues 
with students in depth. Biis enjSiasis on thinking and content 
e}5)loration leads hi^ scorers to reduce content coverage ccarpared. 
to lower scorers, and to voice greater objection to external 
sources of coverage pressure. 

With respect to teachers' conceptions of thinking, hi^ 
scorers _ offered lengthier, more detailed, and Koro elaborate 
conceptions than lower scorers, thou^ scsne of the professional 
language used by each groi^) v^s very -similar (i.e., content,- 
skills, or dispositions). it is int resting to note that the 
gixup possessing more elaborate concepti.:3is of thinkii^ also place 
greater emphasis on thinking as an instructional goal and on depth 
of coverage to better insure thoui^tful exploration of subject 
matter. But the exact relationship between teachers' conceptions 
of thinking, and their instructional goals and resolution of the 
coverage dilemma is unclear. 

Acknowledging, again, the esqploratory nature of this study, 
four reccsmaendations for teacher education can be suggested frm 
conclusions drawn above. Teacher educators need to find ways to: 
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a) incsrease teadiers' cxmdtinent to and ratiomle for 
pixiooting students' thinking as a priumy instructional 
goal; 

b) enhance teacShers* conc^jtual xanderstanding of thinkiiig; 

c) increase teachers' sense of influence with low achieving 
students by changing teachers' perceptions of these 
students; and 

d) incroase teachers' understanding of the positive 
relationship between d^3th or c7>/erage and the proaotion 
of thinking 
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Table 1 



Ocs^eri^ai of Bacia^raixnd CliayBcteristics 
and Qrgardzaticral DimensicYis 



Hi cfli Scorers lower Scorers 

— ^Sex: Male 4 4 

Female 1 1 

— Years teachijig social stiMies 14.6 17.0 

— ^Education: Masters' degree 1 2 

^festers' plijs credits 4 2 

Doctorate 0 1 

— Job satisfaction: Most of tun? 4 4 

Sane of ti3one 1 1 

— ^Sijbject of course observed: 

History 3 4 

Social Science 2 0 

'Social ;:.tudies 0 1 

— ^Nuittoer of schools represented 3 4 

— Teacher load: Teaching periods per day 4.6 3.6 

Average class size 28*6 26.4 

Total nuntoer of students 131.8 97.6 
Ncinber of course 

preparations/day 2.2 2.4 

— Class coitp^ition: % Black 12.6^ 28.8 

% Asian 5.0 1.2 

% Hispanic 11.0 12.8 

— ^Prior social studies adiievemen^ of 
students in the observed class: 

High achievers; 1 3 

Diverse range of achievers 2 1 
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TABLE 2 
HIGHEST rRICXIEK GCftL 

Hicdi Scorers Lower Scorers 
-develop caritical thinkirv and prc4)lem 4 2 

solving abilities 
-teach facts, conc^jts, and theories of 1 2 

history and the social sciences 
-teach past and present prc±)leins and issues 0 l 

faced fcy the U.S. and World 
-teach social values and foster citizenship l i 
-develcp creative thiriKing abilities 0 0 

-develop reading and writing abilities 1 0 

•"develcp discussion skills 0 0 

-teac^ students hew to stu^, take 0 0 

notes and learn 

•develop self-confidence and self-esteem 0 0 

-other 0 0 

TOflftL 7* 6* 
*Note: sosne teachers cited more than one hi^est goal. 

TABLE 3 

Hicfti Scorers Lower Scorers 
-develop critical thinking and problem 5 3 

solving abilities 
-teach facts, concepts, and theories of 3 4 

history and the social sciences 
-teach past and present problems and issues 0 0 

faced by the U.S. and World 
-teach social values and foster citizenship 4 3 
-develop reading and writing abilities 2 2 

-develoj^ creative thinking abilities 0 0 

-develc^ discussion skills 0 0 

•"teach students hw to stucfy, take 1 o 

notes and learn 

-develop self-confidence and self-esteem 0 0 

-other 0 0 

OaHtE 4 
TSMSER INISRBST 

Hicfli Scorers Lower Scorers 

Would you say that in general ejqposing 
to subject iratter content is: 

a) more 0 i 

b) equE^ ly 0 3 

c) less 5 1 
interesting for you as a teacher, as 

develc^ing students' thou^t and 
reasoning processes? 
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Table 5 

TmriTRi" Satisfaction 

DBEtLVED FBO/L: Hicfli Soorers Lcwer Soorers 

— ^Students Thinking 5 1 

— ^StucJents Responding 1 5 

— Working with Ctolleagues 3 2 

—Lesson Planning 1 1 



Table 6 

TeexisBC A^^^ssBSsnt of Botesitial Rrirriers 

Extremely Negative Extremely Positive 

1 2 3 4 5 
Hicgi Scorers X Lower Sotuers 

-Large total # of students 1.3 1.2 

-Large class size 1.8 1.6 

-largt. student course load 2.4 2.4 

-Short 45 iriin. class period 2.8 3.0 
-Large # of graduation 

requirements 3.0 2.8 

•4--5 class sessions per v/eek 3.6 3.6 

-Student ability grocping 2.8 3.8 



Table 7 

Ttarae Jbst Inhibiting Barriers 
(# of teadiers citing a particular barrier) 

Hicfli Scorers Lc Ker Scorers 



— ^Large number of 55tudents (load) 


3 


4 


— ^Ixirge class size 


3 


3 


— Content coverage pressure 


5* 


2 


— ^Lack of planning time 


1 


2 


— Short 45 min. class periods 


1 


1 


— ^Student ability qraupiDQ 


1 


0 


— ^Large # of classes per day 


1 


0 


— ^Diversity of students' abilities 


0 


1 


—4-5 class sessions per week 


0 


1 


— ^Lacfc of quality lesson materials 


0 


1 


15 


15 
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*^Jt)e actual number of lii^ scorers citing coverage pressure is 4^ as 
or*e scorer mentioned coverage pressure twice (i.e./ district 
course-guideline coverage press\ic:e and. coverage pressure resulting from 
the state exam) . 
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Table 8 

Oihree wisiis to Proonte Thihldng 



Hicfli Scxprers Lower Scorers 

— Fe^^x students 4 4 

— ^Freedom to cover less content 3 l 

— Better curriculum materials i 3 

— Staff develcpBsnt to iirprav^e 2 2 
ny teaching 

— ^Psaid leave of absence to think, 2 i 

read, or develcp curriculum 

— 'Jteam ttadiing 0 1 

— Teaitv/grotqp jplanning' 0 l 

"—More lesson planning tijne 0 1 

— ^Peer conservation 1 0 

— Fewer classes per day 1 0 

— Dif fer^it grot^) of students 1 1 



Table 9 

Nature of Ocrverage Oanflict 

Hich Scorers *Iower Scorers 
External source (breadth pressure) 5 2 
Internal source (breadth pressure) 2 i 

*Ihe responses of two lower scorers could not be categorization as 
either external or internal. Further probing revealed this was due to 
the fact that both teachers are from a school participating in a ^top- 
down' district-wide program of curriculxim revision. The two teachers 
receive and acc^.t the district's curriculxim mandates and therefore 
e>5)erience imidi less the dilemma of content coverage inherent in 
planning efforts. See tiie response or "Uoyd"' later in this section 
for details. 
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Table 10 

ErimRTy Sguecb of Breadth of Gbverage Pressure 



Hicfti Scorer. Irwer Scorers 

Internal 

— ^myself 1 3 
External 

— D^xtioent colleagues 0 0 

— D^XuTtment head 1 0 

— ^School administration 0 0 

— District tests or guidelines 1 1 

— State tests or guidelines 3 0 

—Other: AP Exam 0 1 

Total 6* 5 



*Qne hi^ scorer cited 2 primary sources of pressure (i.e., district 
guidelines and state tests) . 



Table 11 



Teachers' Peroe|)tioar^ of the Three Host 
Bixsrtant EcK±ors for the Perf omanoe 
of Above Average Students en Ihihidng Tasks 



Hich Scorincf Teachers 
C3onfidence/self-esteem (4.6) 
Motivation (4.6) 

Learned thinking abilities (4.2) 
Cognitive capacity (4.2) 
Teacher-student rapport (4.2) 



la^r Scorinof Teachers 
CSognitive capacity (4.6) 
MotivatiOT (4.4) 

Learned thinking abilities (4.4) 



Note: Ihe mean score across all 12 factors was qua.te similar for both 
grcfops of teachers; 3.81 for hi^ scorers and 3.83 for Icwer scorers. 
Ihe difference in mean score between the two groups of teachers on any 
given factor did not exceed .6, except "self-confidence/esteem" vAiere 
the mean for hi,^ scorers was 4.6 vs 3.6 for Icwer scorers. 
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Table 12 

Teachers' Peroqrtdcns of the Three Most 
Iii|)Qrtant Factors for the Berf armance 
of Belcw Average Students en ahiriking Ta^ks^^ 



Hicfli Scorim Teadiers 
LearT>ed thinking abilities (4.6) 
Confidence/self esteem (4.6) 
Motivation (4.4) 



Lcwer Scxprincf Teadiers 
Mati\'?ition (4.6) 
Confidence/self-Esteem (4.2) 
Teachers-student rappDrt (4.2) 



Note: Ihi^ mean score access all 12 factors was identical for both 
grc^jps of teachers; 3.83 for hi^ scorers and 3.83 for lower scorers. 
The difference in mean scoire bebveen the two grotps of teachers on any 
given factor did not exceecl .6. 



Table 13 

Teachers' I^cceptims of the Ihree Eacbors 
of Perfarmanoe Tbey Qun Hbst Influenoe 
With jflajv e Averacje Stajdrnt-Ohiiitoers 



Hicfti Soorincf Teachers 
Student kncwledge base (3„8) 
Ihinking skills/abilities (3.6) 
Teacher-student rapport (3.4) 
Writing skills (3.4) 



Lower Scoring Teachers 
student knowledge base (4.2) 
Teacher-student rapport (4.2) 
Writing skills (4.2) 



Note: Ihe mean score acrc^s all 7 factors was 3.20 for hiyli JMon^rs 
and 3.69 for lower scorers;, or an average of about .5 higher for low^ 
scorers on each of the 7 factors. 



Table 14 

Teadiers' Bsrccjptions of the Three Eactors 
of Berfanscinoe They West InQusnoe 
With Celgy Average Sbident-Qiiiikers 



Hicdi Scorincf Teache rs 

Teacher-student raK»rt (4.2) 
Student knowloJge base (4.0) 
Thinking skills/abilities (3.8) 



Lcrver Scoring Teachers 

Teacher-student raj^ort (3.8) 
Self-confidence^ esteem (3.4) 
Thinking skills/abilities (3.4) 



Note' The mean score across all 12 fac:tors was 3.61 for hi^ scorers 
and 3.08 for lov^ scorers, or an average of a little more than .5 
lii^er for hi^ scorers on each of the 7 factors. 
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Table 15 



rectors Hi^ and Ldwst Soorers Agree Aeb Host Inportant 
For Stxsdsant Perforrnance on Blinking Tasks 

Above Average Stixjenb-^Hiinkers 
— ^thinkiiig skills/abilities 
— inherent cx^gnitive capacity 
— ^motivation 

Beloi^y Average Student-^ttiinkers 
"self-confidence/self-esteem 
—motivation 



Table 16 

Factors Hi^ and lower Scorers Agree 
Can Mcsst Inflvisxae With Students 

Above Average Student-Thinkers 
— ^knowledge base 
—teacher/student rapport 
— writing skills 

Below Average Student-Thinkers 
— ^thinking skills/abilities 
— ^teacher/student rapport 



Table 17 

BreaLk3&»n of the IJhsiter of Teachers Using Dispositional, 
Skill, and Oontent language to Describe Their 
Oonoepticn of Olmiking 

Hicdi Scorers Lower Scorer s 
— Dispositional 3 5 
—Skill 5 4 
— Content tmderstanding 1 3 



Table 18 

Breakdoun of the ISaaixac of Teacf ^ers Using Dispositicnal, 
Skill, and Gbnteni: language to Discuss Hov 
Their Lessons Pcosate Thinking 

Hicfli Scorers Lower Soorers 
— Dispositional 0 1 
—Skill 5 4 
— Oontent understanding 5 5 
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CLASSROCXI H»Cn:GES OF HIGH AND U3HER SCORim aS^GHBRS 

Joe OnoskD 

Biis chapter presents findings on the classroom practices of hi^ 
and lower scorlj^ teachers at promoting students' thinking throu^ 
thou^tful classrocan discourse. Hie intent here is to better 
understand the ways in \Aiich the classroan practices of the two grocps 
are sijtiilar and different. Cottparisons are r^rted between the two 
groiqos on 10 diinensions of thou^tfulness using the observational 
scales described earlier. Frequencies are edso reported on certain 
generic teachir^ practices such as the type(s) of: a) reading itaterials 
used; b) writii^ tasks required; and, c) instructional formats eitplqyed 
during class » Ihis infontation will be supplemented and enriched in 
two ways. First, exaitples of observed lessons with a hi^ and a low 
score on each of the 10 dimensions of practice are described. And four 
entire lessons (2 from each groap) are summarized to provide a better 
understanding of differences that exist between the t;X) groups of 
teachers across ittultiple dimensions of thou<^tful practice. 

CEHEfiRING KEGH AND lOWER SOQgM^ W 10 DIMENSIOMS OF TEIJOGHTFOINESS : 

Hi^ and lower scorers can be conpared acrxDss three different 
lesson sets: a) all lessons; Ij) teachers' most thou^tful or best 
lessons; and, c) teachers' less thc«i^tful or remaining lessons. "All 
lessons" refers to the 9 classrc. i observations of each teacher over 
the course of the 3 site-'/isits, or 45 lessons per groi^).! A subset 
of all lessons are teachers' "best" or most thou^tful lessons. 2 
These lessons were used to select teachers as either hi^ or lower 
scorers at promoting thou^tful classroom discourse. ^teachers' 
"remaining'^ or less thou^tful lessons refers to those lessons not used 
to select teadiers as either high or lower scorers. Because the 
findings reveal that with respect to teachers ' remaining lessons 
sizeable differe^ices exist on the exact same scales as that found 



^ Due to illiiess, snow day, etc., one hi^ and one lower scorer 
from each graap was observed on only 8 occasions^ reducing the lesson 
sample from 45 to 44 for each groqp. 

2 ihe teachers with the greatest number of lessens receiving a 
hi^ score of "4" or "5" on the six scales representing the minimal 
requirements of a thou^tful lesson (i.e., HOEAV) were designated hi^ 
scorers (except for "giving students time to think" \Mch required a 
score of "5"). Hic*L scorers' lessons here constitute their best 
lessons. Lower scorans, due to the ):act that none of their lessons met 
the criteria used select high scorers, comprised the 5 teachers with 
the fewest number of lessons receiving a moderately hi^ score of "3" 
or more ai each of the ninlmal requirement scales (except for "tire to 
thijTk" which required a score of "4"). Lower scorers lessons here 
constitute their best lessons. 
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across all lessons the ocatparison of hi^ and lower scorers will 
focus on a) all lessons and b) best lessons. 

With respect to teachers' best lessons and the first 6 scales 
constituting the lodjiimL requirements of a thou^tful lesson^ our 
primary interest is not vftiether differences exist beta^een the two 
grcAjps, tut vdiether the differences are sizeable or meaningfully 
different. Hi^ scorers, by definition, will outscore lowei: scoreirs on 
these 6 EJcales because the initial identification of teadn^rs as hi^ 
or lower sc^c-rers was based on their respective perf ormanoa on the 6 
ininimal recjuirement scales.^ Because teachers' best le&^sons are 
included in the set of all lessons, hi^ scorers are also likely to 
outscore lower scorers across all les5K)ns on the above 6 dlitiensions. 
Hierefore, oar prinary interest is vdiether or not the magnitudes in 
favor of hi^ scorers are si'ieable or educationally significant. 
Teachers will also be ccgpfpared on the remaining 4 diitiensians of 
thcu^tful practice. Because these dimensions were not xased to 
initially identify teachers as hi^ or lower scorers, it iremains to be 
seen what grojp differences mi^t emerge on these scales, and whether 
these differences are sizeable or educationally significant. Finally, 
we will check to see if hi^ scorers consistently facilitate thou^tful 
disocxir^ Hore so than lower scorers. That is, are differences 
observea between the two groaps on teachers' best lessons maintained 
across all lessons? 

To summarise, our analysis of the 10 dimensions of thou^tfulness 
can be stated in the form of three questions: 

1) With tBspect to all lessons and teachers' best lessons are there 
sizeable differences between hi^ and lower scorers on the first 6 
dimensions of practice constituting the minimal requirements of a 
thou^tful lesson? 

2) with respect bD all lessons and teachers' best lessons how do hi^ 
and lower scorers costpare on the remaining four diitensi^ns of 
thou^tfta practice? Are there sizeable differences, and if so, in 
favor of vMch grcfop? 

3) Do hic^ scorers consistently facilitate thou^tfUl discourse more so 
than lower scorers; that is, are there sizeable dif: xences between the 
two grcx?>s regardless of lesson set analyzed? 



^ mrpar^ing all lessons with remaining lessons in Table 4 re-zeals 
that siz^eable differences ocxair between high and lower scorers on 
identifiable scales except for "teacher encourages originality." 

4 This is true only with respect to teachers' best lessons as only 
these were used to identify teachers as hi^ or lower scorers, it 
remains to be seen vdiether or not high scorers outscore Iot scorers 
across all lessons. 



IV - 3 



Tables 1 throu^ 3 provide mean and stanilard devriatioi> ccaparisons 
between hi^ and lower scorers on each scale across the three different 
lesson sets, as well as the "degree of difference" that exists between 
the groups. Olie degree of difference on any given scale is caiputed by 
subtracting the roean of Icwer scorers from the inean of hi^ scorers and 
then dividii^ by the pooled stanc^ard deviation. Typically this 
statistical catputation is called the "effect size". Since this study 
does not involve control and e:xperiinental groups to determine the 
"effect size", this statistic is referred to as the "dogree of 
difference." Using a conservative estimate by e35)eriinental stanaaitls, 
differences exceeding 1 standard deviation between the two grxxps on 
any given scale will be considered sizeable or ineaningful. 

As an exaitple, across all lessons the degree of difference between 
hi^ and Icwer scorers on the scale "students giving reasons" is 1.28 
standard deviations. Assuming the data is nomally distributed, this 
means that if a tea,cher were to inove frcsn the rt^m score of the low 
groap to the inean of the hi^ groip, it would ^ . eqiiivalent to itx^viug 
froKi^ the 50th percentile to the 90th percsentlve of all the teachers 
combined. Frequency distributions can also provide a sense of the 
rather large differences that exist between the two groips of teachers. 
Again using the dimension "students giviiig reasons" as our exanple, 41 
of 44 lessons of hi^ scorers or 93% received a score of or greater 
on students giving reasons whereas only 19 of 44 lessons of lower 
scorers or 43% received a score of "S" or nore (see Table 5). A 
sizeable or uveaningful difference clearly seems to exist between the 
two groups on students givii^ reasons, but note that the 1.28 degree 
of difference is not particularly large viaen OOTpar^d to the 
differences that emerge on other d^Tnensions of thou^tful practice 
(see Table 1) . 

Presentation of f irxiings on each dimension will be followed by an 
offioiple of a lesson receiving a hi^ or outstandii^ score (i.e., "4" or 
"5") and an exaiiple of a lesson receiving a low score (i.e., "1" or 
"2") . In cases x*iere the degree of difference, is sizeable and in favor 
of hi^ scorers, the outstanding exairple will be taken frm a hi^ 
scorers' lesson and the non-exenplary or low scoring tixanple will be 
talcen from the lesson of a lower scorer. On scries vAiere sizeable 
differences between the two gmjps failed to emerge, or on scales; vftiere 
sizeable differences were found bixt both groips scored relatively low 
(i.e., a groip mean of less than '3'), only an outstanding example will 
be provided-^with the cibvious inference that lessons receiviixf a low 
score failed to manifest the dlinension under discussion. 

Few Topics Discussed 

With respect to all lessons and tochers' best lessons, sizeable 
differences were found between hi^ and lower scorers on the dimension 
of '"few topics". A high score indicates that few topics were addressed 
during a class session vdiile a Icw^ score indicates that many topics 
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were covered. TSie degree of differeiK^e between the two groi?>s 
exceedjed 1 standard deviation for both lesson sets. 

As an exaitple, one hi^ scorer f-xaised class discussion on the 
problem of creating material v/ell-being for everyone in society by 
analyzing the following prc^x>sition: 'We live in a world oC limited 
resources but are creatures of xiailimited desires." An exaitple of a 
lesson where many topics were discussed can be fcuni in a lesson on the 
abolitionist movement in the United States. Discussion ranged across 
such topics as relations between the U.S. and the countries of Iran, 
Soith Africa, and Nicaragua, and how the abolitionist, wcaasn's ri^ts, 
and prison reform movements all fit into Jacksonian democracy. 

Substantive Cdierence of Lesson Content 

large differences appeared between hi^ and lower scorers with 
respect to all lessons and teachers* best lessons. Ihe degree of 
difference between the two groi^js approached 2 standard deviation for 
both lesson sets. 

An exanple of a hi^ score on substantive cdierence can be 
witnessed in a hi^ scorers' lesson on the nullification debate betv;een 
South ^ Carolina and the faieral government during the Jackson 
administration. Ihe class first discussed the meaning of the term 
nullification, llhen arguments for nullification as presented by vice- 
president John C. Calhoun were e^lored, followed by an analysis of 
president Jackson's counter argumfeiVcs. Once the argimaents on both 
sides were clearly understood, students were asked to generate 
hypothetical situations today where die nullification issue could 
arise and vdiere a state mi^t be justified in its actions o 

A IcM score on substantive coherence can be observed in a lower 
scorers' lesson on president Theodore Roosevelt. The lesson began with 
students generating criteria or qualities of "leadership greatness" to 
help them assess T.R. »s leadership as president. Students then began 
to provide evidence from their readings on T.R. relevant to the 
criteria generated, but suddenly the central question changed from 
leadership to Aether or not T.R. should be considered a "great 
Progressive". Shortly thereafter the teacher again redirected 
discussion by asking if beir/g a reformer is innate or can be learned. 
Por^ the Kxraainder of the period, discussion switched frcm T.R. «s 
position on trusts to a discussion of his leadership greatness and 
then back again to his status as a great Progressive. The bell rang 
with no attenpt at closure on the teacher's part. 

Students Given Time Before Restx)ndina 

Across all lessons both hi^ and lower scorers gave students 
sufficient time to think before responding, vdiereas a sizeable 
differences in favor of high scorers emerged with respect to teachers* 
best lessons. Examples of exenplary and non-^enplary lessons are not 
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provided bscause hi^ and lower scorers do not differ across all 
lessons on this diitvension, and because "giving students time" is self- 
explanatory. 

Teacher Provides Students With Challenqincr T&sks 

Across all lessons hi^ scorers provide students with itore 
challenging tasks than lower scorers, with the degree of difference 
again approaching 2 standard deviations. However, with respect to 
teachers • best lessons differences are less pronounced and fail to 
exceed 1 standard deviation. i^parently, hi^ scorers Hore 
consistently provide challenging tasks for students, but the degree of 
challenge is about the same with respect to teachers* best lessons. 

An exanple of a hi^y challenging task can be seen viien a hi^ 
scorer had students assess \diether or not Andrew Jackson's presidency 
enhanced or diminished tlie level cf democracy in the United States. 
Using criteria of democracy the class had developed in a prwious 
lesson, students had to identic, describe, analyze, and assess various 
aspects of the Jacksonian presidency (e.g. , the election, the 
inauguration, the spoils system, his view of the presidency, etc.) for 
its democratizing or non-democratizing effect on the U.S. 

A lyei score on challer^e can be observed in a lower scorer's 
lecture-recitation class session. Students were occasionally asked 
recall type questions to indicate their retention of material presented 
in the text or by the teacher. Many terms were defined such as 
inpressment, intervention, ferbary pirates. Leopard Incident, etc. Ihe 
teacher also outlined for students the views of past presidents on the 
issue of isolationism. 

Teacher Models Thoughtfulness 

Across all lessons and best lessons hi^ scorers nvodel thou^.tful- 
ness more than lower scorers. Differences exceed 1 standard deviation 
with irespect to both lesson sets. 

An exanple ^ of a hi^ scorer modeling thou^tfulness in an 
exenplary fashion can be witnessed in a lesson on the rise of 
anti-Semitism in Germany between 1918 (Treaty of Versailles) and 1941 
(the beginning of the Holocaust) . The teacher modeled thoi^tfulness 
in the following ways: 

-cortplimented studento at the beginning of the period on their 
thinking during ye^c^rday's discussion 

-acknowledged that the textb^k presents many details without a 
unifying structure, making it difficult to determine \*at information 
is inportant 
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-ir^isted that students give e5q)lanations and reasons for their 
coacnents, and vdiere appropriate would direct the class to the section 
of the reading vrtiere the evidence could be found 

-encouraged students to recall information and ideas without their 
notebook, but if unsuccessful, they could then refer to their notes 

-acdcncwledged the difficulty of defining the terra "statistic" and of 
appreciating the magnitudo of such nuinbers as 1 or 10 million lives 

-ccsrplimented the reasoning sequence underlying a student's answer 
even if the answer was incorrect (e.g., becavise the student did not 
possess scans critical piece of infonration) 

-ccsnplinented students when good questions were raised 

-emphasized ownership of ideas and questions by students (e.g., 
"Jeff's question" or "Sue's position") 

-ejdiibited certain mannerisms associated with thou^tfuli^ss (e.g., 
eyes looking toward the ceiling in an act of reflection, head cocked 
at an angle to indicate a more careful listening posture, stroking of 
the chin \diile deliberating, enthusiastically nodding his head while 
student speaks, etc.) 

Much less modeling of thou^tfulness can be observed in a lower 
scorers' lesson on Frederick Jackson Tun^ and his "Frontier Thesis". 
The aim of the lesson was to have students consider the following 
question: "Hew has the Frontier Thesis influenced the development of 
America?" Teacher modeling was deficient in the followii^ ways: 

-the central question itself was vague and open to various meanirgs 
and interpretations (e.g., how can a thesis developed long after an 
historical period influence the development of that period?) -class 
discussion strayed from the central question without direction or 
purpose 

-the teacher did not atteirpt to connect student comnvents back to the 
central question 

-the teacher did not share with the class his thinking about any of 
the points or issues raised by students, or about the central question 

-the teacher never sustained his devil's advocate role beyond an 
exchange or two, resulting in a sdrdzophrenic advocacy of ccatpeting 
and at times contradictory positions that precluded an analysis of 
statements and positions 

-the teacher tended to focus on students' concliasions rather than on 
the reasoning for their conclusions 
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-there \i^as no closure or summary of the ideas discussed during the 
lesson 

-the lesson was essentially a "sha^ and tell" of ideas on the part of 
students, with no attenpt on the teacher's part to orchestrate a 
discussion eadiibiting itcfvement and purpose 

Students Give Reasons and Explanations for Stateinents 

Across all lessons the students of hi^ scorers offer reasons more 
consistently than do the students of lower scorers. with respect to 
teachers' best lessons, however, the means of the two groips are very 
similar and the degree of difference correspondingly small. Recalling 
that 3 of 5 Icwer scorers' classes were cortposed prmarily of hi^ 
achieving students corrparaa to only 1 of 5 among hic^ scorers, it is 
not surprising that on certain lessons 'reason giving' by lower 
scorers' students is coirparable to that of hic^ scorers' students. 
Overall, though, hi^ scorers more consistently require students to 
provide reasoning for statements. 

An exaitple of a lesson vSiere students consistently si5:poi±ed their 
statements with e^qjlanations and reasons is found in a hic^ scorers' 
econaaics class. A student responded to the teac±ier's yes/no question 
with a response of "no". Ihou^ the answer itself was correct, the 
teacher did not comment. Instead he raised his ri^t arm and rotated 
his index finger in a circular fashion. Instantly the student blurted 
out "because!", and after mconnentary pause for reflection he proceeded 
to give reasons for his "no" response. Ihis incident illvistrates the 
extent to vMch students had been trained to provide si^pporting 
evidence for their statements, as a sinple hand gesture by the teacher 
could new prarpt the desired behavior. 

lack of si^jport for statements can be observed in a lower scorer's 
Icicture-recitation lesson. Ihe teacher reviewed for students 9 social 
problems attacked by Progressives (e.g., child labor, factory safety, 
education, etc.), as well as the solutions Progressives proposed and 
the extent of their success. The students were required to follow 
along with a worksheet and write down each of the prt^blems, the 
proposed soliitions advocated by Progressives, and the relative success 
of the proposals as articulated by the teacher. 

HOIAV — Minima l Rgqin'rg gnents of a Thoucflitful Lesson 

Because of the rather large differences observed between the two 
grca^ on inost of the above scales with respect to both sets of 
lessons, it is not surprising that sizeable differences, again in favor 
of hi^ scorers, emerge between the two groi5)s on HOIAV. 

We new turn to the remaining four dimensions of thou^tful 
discourse not used to select teachers as either hi^ or lower scorers. 
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Careful Oonsideration of Reasons and Explanations 

Acaxss all lessens a large differerK:e in ineans emerged between 
hi^ and lower scorers in favor of hi^ scorers, with the degree of 
difference approaching 2 standard deviations. Similar findings ernerged 
for teachers' best lessons. 

An exairple of a hi^ score on carefaJ. consideration . can be 
witriessed in a discussion of the factors that led to a large U.S. 
national debt in a hi^ scorers' economics class cooooposed of a diverse 
range of students. Each potential factor was carefully analyzed to see 
if it could indeed help cause the national debt. An exaitple e»3hange 
between the teacher and the class mil hi^i^t the diinsnsion of 
careful consideration of ideas and reasons. Having agreed as a class 
that an increase in pc^^ulation could contribute to an increase in debt, 
the class begins to consider "increase in average age" as a potential 
factor: 

Teacher: VJhy would an increase in average age cost ito?:e? 

Student: Because more people are staying around more. Because there are 
more people and so you stay around longer. Then there are more 
people and that way more people get social security and other 
things and you have to pay for these people because they're 
still alive. 

T: O.K. So it's not only that there are more of them, but they're 
different. How are they different? 

S: Well, they live longer. 

T: Airi^t, they live longer, so? 

S: So they collect more money. 

T: Why should they get roare money than younger pec^le? Because they're 
old and wrinkled and have white hair (pointing to his own)? 

S: Becaijse they worked alreacfy. 

T: Ihey worked already. So now vAiat? [Pause— no student responds] 
Scxneone's got to help lis out here. She says there are more of them. 
But we alreacfy know that because there are 240 million today rather 
than 76 million in 1900. But it's only half the story that the 
pofwlation got bigger. But it also got older. 

S: What was the average age back then and now? 

T: Life ejqjectancy then was roughly 56 years; ard now for the entire 
population around 73 years. So, yes, there are more older people, 
the average age of the country is a lot older, especially in the 
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last two decades. [Pause] So, v*iat does this mean in terras of our 
overall question? Why are the government's e>q)enditures goijyg vqp? 
Hew does the fact that they're getting old drive 15) eJ5>enditures? 

S: Ihe government has to pay for that. 

T: For \*-at? 

S: Their house, their 

T: They've already paid off their honve mortgage. They're retired. 

S: Their medical, and they go to the hospital so many times. 

T: O.K., that's two. See you knew all the time. Alri^t, what else? 

S: The average retirement age hasn't changed and they'll have a longer 
time to receive social security benefits, also there are more people 
of a working age v*io will receive benefits. 

T: O.K. Let's put this on your list of factors as, 'Population grows 
and ages. ' 

Failure to engage students in careful consideration can be 
witnessed in a lower scorers' lesson with honors students. The topic 
for the day was political leadership, and students were to 
individually select v*iat they believed were the 3 most iirportant 
leadership tiraits from a list of 5 on the blaciWx)ard (e.g., charisma, 
moral integrity, political savvy, etc.) and to add one of their own. 
Discussion began with students stating the trait they added, and then 
a lengthy hand coiuit tally was taken to determine vMch 3 of 5 traits 
received the most votes. Because students were not required nor given 
the opportunity to provide reasons for their selections, careful 
consideration of reasons was absent. Following the hand count the 
teacher for the remainder of the lesson ov'arviewed with students an 
x^xxaning assignment. 

Teacher Enaaaad Indivi.dual Students In Socratic Dialogue 

Though neither groi^ consistently scored hi^ on this dimension, 
sizeable differences emerged between the two groups in favor of hi^ 
scorers across all lessons and teachers' best lessons. The degree of 
difference exceeded 1 standard deviation for both lesson sets. The 
criterion for a Socratic exchange was 6 unintem^rtzed exchanges, 3 
each, between teacher and a student (or between two students) . Only 
an example of a lesson scoring hi^ on this dimension will be provided 
because it would not be instructive to summarize a class vftiere such 
exchanges failed to occur. 

An exairple Socratic dialogue occurred in a hi^ scorers* U.S. 
History class carprised of low achieving students. Students assumed 
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the role of U.S. senators on a Congressional cxxnmittee interrogating 
Andrew Carnegie (role played by the teacher) about his loonc^listic 
business practices. One excihange between Carnegie ard student-Senator 
Mbrris was zs follows: 

Carnegie: Senators I am a very, very rich man. What do I ami 

Morris: You cwn nearly everything... you own steel, ceirent, machinery, 
railroads, and mills. 

Carnegie: I own nearly everything but not everything. But I have 
worked hard for it. I have earned it. Don't tell ms I am 
wrong to cmtx it. I worked my way yjg from the poor little kid 
destitute on the streets. . .and now I am proud of \^t I won. 
I own 75% of the steel industry vAiy shaildn't I have 75% of 
the market? Why shouldn't I have 75% of the territory? 

MDrris: Ihat*s not fair. That's uriscnpiLous. 

Carnegie: What do you mean ijjnscnpalous Senator. Haven*t I earned 
everything I own? 

Morris: You would be able to eventually am everything and ndbodi'' 
would be able to have a business. 

Carnegie: But vftiat do you mean? If I could own all of industry why 
shouldn't I? Isn't this a free country? Isn't oornpetition 
v*iat it is all about? Don't I pay my workers and set fair 
prices? 

Morris: But nobocty could ccaoopete with you and I don't think there 
would be any business after that for anybody but you. 

Carnegie: Senator you are a difficult man. You are really a difficult 
man. . . 

T eacher Invites Original Rest)onses 

Ihe mean scores for both groi^js on both sets of lessons are low. 
Only 13 of 44 hi^ scorers' lessons and 5 of 44 lower scorers' lessons 
received a score of "3" or greater on this din^nsion of thou^tfulness. 
Die degree of difference was not sizeable on either lesson set. 

One of the 3 lessons in the entire sanple of 88 lessons to score a 
"5'" occurred in a hi^ scorer's freshman social studies course. The 
intent of the lesson was to shw students hew a society's values 
influence its political, econcanic, and social arrangements. The lesson 
began with a teacher led discussion on vdiat it means to say every 
culture has certain values. To highli^t this idea students v^re asked 
to consider hw two political systems would differ if one valued 
equality and the other did not* Thou^ this task invited original 
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responses frcsn students, the*, task that followed require them. In 
small grocp discussion each group was to construct and describe the 
wrkings of a subsystem or aspect of an imaginary, ideal society 
(e.g. , its educational, medical, tax, system) . A set of questions was 
provided by the teacher to help direct each grocp-s discussion (e.g., 
Vfould everyone be taxed the same amount? VJhet would be taxed? Would 
irdividuals and corporations be taxed at the same rate? etc. ) . 

Teacher Reveals That Disagreement E?dsts Between Authoritative 
Sources 

Here again both groips scored consistently low with respect to 
both lesson sets, but with respect to teachers' best lessons, the mean 
of hi^ scorers was 1.46 vs 2.57 for lower scorers. For the first and 
only time a sizable degree of difference in favor of lower scorers 
emerges; i.e., 1.56 standard deviations. Because neither groap scored 
particularly hi^ on this diinension only an example of a lesson 
receiving a hi^ score will be providedo Only 5 lessons received a 
score of "5", 4 by hi^ scorers and 1 lesson by lower scorers. The 
following exairple is taken from a hi^ scorer's lesson. 

In a psychology lesson, students critiqued Carl Rodgers' "client 
centered" therapeutic approach from both a Freudian and a Jungian 
"therapist centered" perspective. Students were made aware that 
e:qperts within the field of psychology disagree fundamentally on 
treatment methods, including assuirptions about the abilil^ of the 
patient to heal hiiVhersaLf, about the need to delve into the 
patient's past, about the need for a forceful, directive therapist, 
etc. 

Summary 

What do the above findings suggest witli respect to the three 
questions focusing our analysis? To each of these questions v/e now 
turn. 

1.) With respect to all lessons and teachers' best lessons are there 
sizeable differences between hi^ and lower scorers on the first 6 
dimensions of practice constituting the minimal requirements of a 
thou^tful lesson? 

Examining teachers' best lessons, we found hi^ scorers' classroom 
practice to be si?)erior to Icwer scorers' on; "few topics", "content 
coherence", "tiite to think", and "teacher modeling thou^tfulness". 
Sizeable differences were not found with respect to "students giving 
reasons" and "teacher providing challenging tasks". iDwer scorers, at 
least with respect to their best lessons, seem to construct equally 
challenging tasks for students and solicit from students responses that 
reveal their reasoning. However, conpared to hi^ scorers they tend to 
do much less modeling of thou^tfulness, and cover a greater reoige of 
topics during a lesson with much less substantive coherence. More 
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topics covered my ej^lain why students of lower scorers are given less 
time to think. 

With respect to all lessons, the sizeable differences found above 
on teachers' best lessons are maintained exce^Dt for the dimension "time 
to think" • Both graips continue, however, to give students 
sufficient reflection time before respondir^j (i.e., mean scores of 
greater than M'). Substantial differences emerge on two other 
dimensions of practice; i.e., hi^ scorers give students significantly 
more diallenging tasks and much more frequently require that students 
provide reasons for their statements* With the addition of challerijitig 
tasks and the giving of reai^ons, across all lesisons sizeable 
differences betv^een the ta^ groi5)s exist on 5 of the 6 dirosnsicns 
constituting the minimal requiren^ts of thou^tfUl practice. 

2.) With respect to all lessons and teac±ers» best lessons ho^^ do 
and lower scorers coeoopare on the remaining four dimensions of 
thou^tful practice? Are there sizeable differences^ and if so, in 
favor of which group? 

CSonceming teachers' best lessons, sizeable differences emerged 
between the two groi;5)s of teachers on three of the foor scales* High 
scorers to a much greater extent engage students in "careful 
consideration of reasons" and "Socratic dialogue", vrtiereas lower 
scorers are more likely to reveal to students "disagreement anong 
authoritative sources." Recalling that both groups solicit reasons 
fran students with about equal frequency on their best lessons, it 
appears, however, that hi^ scorers are mor^ likely to go beyond the 
sharing of reasons and actually exam the merits of those reasons 
throu^ careful consideration. Careful consideration may e35)lain the 
substantial differences also found on Socratic dialogue, as Socratic 
probes often acconopany scrutiny of ideas and arguments. 

Ihou^ both groi:?)s scored relatively low on the "authoritative 
sources** dimension (i.e., less that "3"), how does one explain lo&;er 
scorers* ca:?)erior performance on this in their best lessons? One 
possible explanation involves lower scorers* greater propensity for 
content coverage and reduced efforts at careful consideration. It may 
be that the expanded coverage sweep of lower scorers increases the 
likelihood that students will be ejqposed, albiet siperficially, to 
differing interpretations and perspectives. Conversely, hi^ scorers' 
enjliasis on d^>th of coverage and careful consideration with respect to 
their best lessons may reduce the likelihood that carpeting 
perspectives will be discussed in a given lesson. 

Across all lessons hi^ scorers again display si:55erior perfonrance 
on the dimensions of careful consideration and Socratic dialogue. 
Icwer scorers, ho^\?ever, do not maintain su|)erior practice on 
"authoritative sources," with neither groqp revealing much of this 
dimension (i.e., mean scores of less than "2") . As with teachers* best 
lessons, teacher solicitation of original responses from students 
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failed to reveal differences between the two groops of teachers across 
all lessons. 

3.) Do high scorers consistently facilitate thoughtful discourse itore 
so than IcMsr scorers; that is, are there sizeable differences between 
the two groi^js regardless of lesson set analyzed? 

itoross all three lesson sets hi^ scorers e>diibit si^jerior 
performance on tha same 6 dimensions, and ccatparable performance on the 
remaining 4 dimensions. ^ Acaxss all lessons sizeable differences in 
favor of high scorers emerge on 8 of 10 dimensions. Even if one 
removes the effect of teachers' best or select lessons on each groip's 
total sanple of lessons by only considering teachers' less-thou)^tful 
or "remaining lessons", all of the sizeable differences remain (see 
Tables 3 and 4) . 

A final check of stability or consistency across lessons is 
performed by ccsiparing the mean of teachers' best lessons with the mean 
of their remaining lessons on each dimension of practice (see Tables 5 
and 6). If the degree of difference on a given scale is found to be 
less than a standard deviation, then substantial differences do not 
exist and we can conclude that the respective group of teachers were 
consistent in their practice on the respective dimension. Among lii^ 
scorers, a _ sizeable difference emerges on only 1 of 10 scales (i.e., 
time to think) . For lower scorers sizeable differences emerge on only 
3 of 10 scales (i.e., teacher modeling thou^tfulness, students giving 
reasons, anc? the questioning of authoritative sources). We can 
therefore conclude that hi^ scorers caijared to lower scorers on most 
practice dimensions consistently display si^jerior performance 
regardless of the lesson set analyzed (i.e., all, best, or remaining 
lessons) . 

C3CMKVRIKG OCg'flPIETE IE SSCM5 OF HEGH AND lOSER SCCRERS ACR36S MJIiriPIE 
DIMEKSKaB OF gmaiTPDIKESS ; ' " 

Having presented quantitative differences between the two grotps 
and provided brief exaitples of lessons scoring either hicdi or low on 
each of the 10 diinensions of thou^tful practice, four r^resentative 
lessons (2 frcan each group) are now summarized to provide a more 
concrete, enccnpassing understanding of the differences in practice 
that e>,!ist between the two grcinp?;. A U.S. history lesson of a hi^ 
scorer is paired with a U.S. history lesson of a lower scorer, and a 
hic^i scorer's non-history lesson (i.e., psychology) is paired with a 
lower scorer's non-history lesson (i.e., civics). Lessons chosen need 
not be controlled for students' prior achievement because a majority 
of hi^ scorers' classes contained students of diverse or low prior 
act. .evement in social studies \diereas laver scorers had a majority of 



^ E5{cept on authoritative sources and teachers' best lessons v^ere 
lower scorers ejdiibit si^jerior perfomance. 
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students with hi^ prior achieveofnent. This firding suggests that 
being classified as a hi^ scorer did not dqpend vopon working witli a 
nore receptive student population. Nonetheless, the paiired U.S. 
history lessons both contain students of hi^ prior achieveiwent. Ihe 
hi^ scorer non-history lesson contains students of diverse prior 
achievement vMle the Ic^^ scorer non-history lesson ccarprises 
students primarily of low prior adhieverrent. 

History Lesson of a High Scorer with Students of Hicfli Prior 
Achievenvent . 

Ihe following lesson is part of a unit on the Constitution in 
Hilary's 10th grade U.S. History class. One question guides inquiry 
for the entire xmit; i.e., '^ere the ftamers of the C3anstitution 
primarily iac±ivated out of self-interest or for the good of the 
country?" At the erd of the unit students must write a paper 
e^qjressing thair position on this question. Students are analyzing 
infonnation about a sample of 11 framers to generate concliasions about 
all 55 frainers. Today's lesson has three goals: a) review the thinking 
skill of "organization" and introduce the thinking skill of 
"synthesis"; b) determine the various political goals of inib'idual 
framers and make generalizations about the framers as a group; aM, c) 
have students write a thesis sentence (i.e., position statement) for 
their x^xxsning paper. 

Hilary begins the lesson by reviewing the 4 steps involved in 
organizirig infonnation learned yesterday. Ihe four steps are: 

1) Identify your goal (i.e., what will you do with the information?) 

2) Survey your data 

3) Determine yaar categorizes of organization based xipon your goal 
4/ Place information in the proper categories 

Sha then asked students if they had aiiy problem with the hoKiework 
assignment in whidi they were to corrplete a chart using the categories 
of organization they had generated in class the day before (e.g* , name, 
occtpation, degree of wealth, hew wealth was achieved, political goals, 
etc.) . Students had problems inferring from the readings the political 
goals of GcxoB framers. Rather "^han provide students wi.th answers, 
Hilary raises questions and offers factual hints until students make 
inferences and reach conclusions. For exairple, students cane to see 
that James Madison feared the self-inteirest of many of the framers and 
therefore desirecf* ccarproanise between the various factions in order to 
stabilize the country and hold it together, or that Edmund Randolph 
wanted a balance of power between the 3 branches of government but that 
each branch be run by members of the educated and monied elite. Hilary 
addressed other questions in a similar manner, specifically students' 
Koral indignation at the thought of greedy conventioners like that of 
Jonathan I^yton: 

Student it Vihy did they have Jonathan Dayton there, he was cudi a jerk! 
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Student 2* 

Hilacy: What do you mean he was a jerk? 

Student i: lie was a crodk, trying to cheat the govemiftant vMle they're 
tryirvg to inake a good government. 

Student 3: Do you think it was because he was a war hero or scxnething? 

Student 4: I'm sure they ail cheated that way, vdiy else would they be 
there? 

Hilary: Who was selecting these people to go to this Constitutional 
Convention? 

Student 5: The people. 

Student 5: Hitemselves! 

Hilary: Sure, they were an elite groip selected by an elite groap. 

Yew have to be able to vote in order to vote for scsaebody to 
go to the Constitutional Convention, So you're dealing with 
a select groap here. We would love to believe that then and 
new all people vHao are involved in politics are honest and 
just but we know that's not the case..." 

Another analysis of a framer's intentions is as follows: 

Hilary: Why would Roger Shenran, if he achieved his position in life 
throu^ his own initiative want as one of his political goals 
to keep the pcwer in the hands of an elite? What do yoa think? 

Student 1: Maybe because he made it on his own he felt he was better 
than the rest of the people. 

Student 2 J He was an exception. 

Student 3: He sort of educated himself. He worked hard to get where he 
was and didn't think people should get to vote just by not 
doing anything. 

Hilary: Alri^t. Anythir^ else? Where had he started? What does he 
think he kiKws about those people? 

Studeiit 3: [munibled-inaudible response] 

Hilary: You see \^t she (i.e., student3) ^ saying? We don't know 
his political goals but we can m?ke scens assun^jtions. One 
assunption Jim is making is that Sherman has already lived 
with these people vSio are not concerned with government 



ERIC 



103 



XV - 16 



affairs and he xncws this. Ihan Sherman's comem is that 
they not assume sudh a position (i.e., political decision- 
making) . What would Shennan assume are the major concerns of 
farmers and day-to-day working class people? ^tot's theiv 
major ocax^em? 

Student 4: Their land 

Hilary: Sure, they've got to survive. Having the benefit of time to 
play politics is a luxury. And mai*>e he is saying that these 
people cannot do this. Ri^t? And therefore politics should 
left in the hand of people v^o are educated to that kind of 
position. 

Having satisfied students' concerns regarding the political goals 
of various framers, Hilary introduces the thinking skill of synthesis: 
"The next step, now that vre've organized all of this information, is to 
synthesize, to put it together to make something new." Varioas 
analogies are tised to clarify the concept such as the ingradients or 
parts that go into making a pizza or the vtole. Ihe three steps of 
synthesis are ejqplained: 

1) Look at your organized information 

2) Decide vAiat is in coaranon 

3) Pull the ccanmon information together into a generalization 

Students are new asked to synthesize or generalize across each 
category of the honework chart starting with "name/backgrxjund". Three 
students contribute and conclude that tiie framers are man, vMte, 
landowners, and voters, with English backgrounds. Generalizations are 
also made regarding "oca^aation", "degree of wealth", and "how wealth 
was gained". The class now arrives at the critical generalization 
with respect to their v^xxming position paper. Hilary asks: "When you 
look at their political goals, vdiat general features can you find in 
the majority of delegates?" One student suggests that the framers 
were out to "benefit the elite", while another student states that 
they wanted to develop "rules and laws that would benefit themselves." 
The class seems to be in agreement here, so Hilary asks if there are 
any exc^jtions. Discussion leads to the conclusion that Madison, 
Franklin, aid Hamilton are the exceptions. 

Students are then given 5 minutes to "take the various cammon 
elements" and contoine them into "a general statement" \Mch will serve 
as the thesis for their position paper. Hilary asks for volunteers to 
share with the class their thesis sentence. Six are read, two of vdiich 
are as follows; 

"The framers of the ' Constitution were interested in developing 
regulations that were beneficial to their wants and needs." 
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"TbB creators of the Constitution were insmbers of the elite class 
representing the elite to benefit the elite." 

With scsne stidents Hilary suggests minor but iiiportant modifications. 
For exanple one student refers to the framsrs as "elite delegates" in 
her thesis sentence. Hilary points out that the phrase "elite 
delegates" duplies that there were also "non-elite delegates" whicii was 
not the case. She suggests the student vse an alternative phrase such 
as "the delegates were part of the elite". 

Before the class ends 4 additional points are adcSressed: 

-Hilary informs students that generalizations do not always contain 
arguments, and then asks students to determine if indeed their 
generalization contains an argument. 

-Hilary offers students a way to check for an argument by assessing 
whether or not there is an alternative position to their own? 

-Hilacy reminds them that they're looking at a saitple of 11 framers 
frcan a total of 55/ ar-d therefore the evida-ice they si;?!ply to si^port 
their generalization must apply to most of the 11 framers or it won't 
be convinciig. 

-Hilary e}q)lains the hoanework assignitent in vAiich students are to 
outline the bocfy of their position paper by providing the ideas and 
facts that will si:qcport their thesis. 

Reviewing the above lesson with respect to the observational for 
scales, many outstanding features emerge. Few topics were discussed, 
specifically the thinking skills of organization and synthesis and 
their application to the question of intent of 11 Constitutional 
framers. The content was presented and discussed in a ccherent 
manner, with an eye always focused on the relationship between the 
data and the overriding unit question. The teacher modeled 
thou^tfulness v*eri summarizing the inductive steps students used to 
arrive at the political goals of various framers, v^ien sharing with 
the_ class her inferences about framers' political goals, and vto 
giving cwi^rship to ideas e}5>ressed by students (e.g. , "one assunption 
Jim is making. . .") . The problems pc^ed were challenging as evidenced 
by the degree of inference requdied. Students' responses were 
carefully analyzed for factual accuracy, correctness of inference, and 
clarity of generalization in their thesis statement. Students were 
given time to think before questioning and responding, and Socratic 
excOianges occurred on occasion between Hilary and a few students vAien 
assessing framers' political goals and students' thesis statements. 

U.S. Higtnry TAggon of a Lower Scorer with Students of Hicfli Prior 
Achievement 
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Larry's year-long Advanced Placement United States History class is 
ccffcpcsed of hi^ achieving juniors. Hie lesson focuses on the 1848 
Seneca Falls Declaration of Wcanen's Ri^ts, and occurs at the end of a 
unit on the broader theme of Jacksonian democracy. Cie goal of the 
lesscxi is to answer the following question: Did American wcsien have a 
case for equal ri^ts during the Jacksonian period? 

At the start of class students received a one page handout vdth 
excerpts frcm the 1848 Seneca Falls Declaration and the 1776 
Declaration of Independence, and a reproduction of a partial chapter 
(i.e., 6 pages) on the ri^ts of women from a U.S. history college 
textbook containing relevant, quotes by Abigail Adams and her son. A 
teacher-centered class discussion began once students finished reading 
the page of excerpts firom the Declarations of 1776 and 1848. 

Discussion opens with Larry asking the class to identify the ways 
in vMch the two documents can be coipared. Students eventually agree 
to have the following summ^jy statement written on the blactooard 
regarding the Declaration of Independence; "People have the ri^t to 
alter or destroy the government", with respect to the Seneca Falls 
Declaration, students agree to the summary statement that "Wcsten have 
the ri^t to refuse allegiance to a government." Larry asked the class 
if these two statements meant the same thing, but the question vras not 
pursued. Instead the class moved on to a second point of ccaiparison; 
students paired "All men are created equal" from the Declaration of 
Ind^jendence with "All men and women are created aqual" from the 1848 
Seiteca Falls Declaration. 

At this point a student, Peter, suggests that Jefferson and the 
other revolutionaries would have been lau^ied at if they included women 
in the document, especially since they were directing 'their nessage to 
British leaders ^o were all men. He adds that the Seneca Falls 
Declaration occurred once "we were our own country, ai>d the people of 
our country were saying men and wcanen are equal." larry immediately 
asks the class if they agree with Peter's statement. IWo students 
offer brief cosnments that seem to disagree with Eteter (inaudible on 
tape) . Larry then jokingly informs the class that "...if I were alive 
at the time you know for sure I wouldn't have badked women's ri^ts." 
Two female students object and call him chauvinistic. Larry qualifies 
his_ comment by stating, "I said in 1848. Now it's 1986." He then 
redirects the class ty requesting other points of cortparison between 
the two documents as "a foundation" is need^ to help them ansvrar the 
central question of the day's lesson. 

A student mentions that both documents allude to a "history of 
repeated injuries." Specific examples of injuries are cited such as 
taxation without r^resentation with respect to the Declaration of 
Ind^jendence, and no vote, no law making power, and no property ri^ts 
with respect to women and the Seneca Falls document. Discourse then 
proceeds as follows: 
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Larry: What's the problem with these injxiries? 

Student: Men would say this is no big deal. 

Larry: What else is the problem, vdiat is the problem with this 
document? 

Student: Oliese injuries are still going on today. 

Larry: Ma:^, bat what else? 

Student: Ohey had no pcwer to enforce it. 

Larry: O.K., v*at else? 

Student: Nobody would pay attention to it. 

lairry: Who wrote the document? 

Student: Wcsnen. 

Student: But rten wrote the Declaration of Independence. 

Larry: cai, there are many inen today v4io come on to the woman's side. 
Not me, but there are men v4io would. But do you find that this 
mi^t have been one-sided this time— very one-sided? In other 
words wcanen didn't have a case for equal ri^ts. 

Larry then asks the class v^t they think. Two students offer 
comments, but neither is a direct response to Larry's statement. He 
then abn:5;tly changes the direction of the discussion, either asking 
two different questions in one sentence or irodifying the initia]. 
question in the latter half of the sentencet "Which ri^t do you think 
was violated the most, vMch affected wcanen probably the la^?" One 
student cites lack of participation in law making. A second student 
mentions lack of hi^ier education, and hew this would result in women 
not "taicwii^ how to present a good arguK^t. Larry immediately asks 
her if that Is a social or economic argument to \Mch the student 
responds "both." He then calls on one more student to share her 
thou^ts before asking the class to turn to their second. handoit. 

A student reads aloud a statement supporting women's ri^ts by the 
son of President Jdin Adams and his wife Abigail to members of the 
Etouse of Representatives in 1838. Olie quote ends with Adams posing 
the following questions: "Are women to have no opinions or actions on 
subjects leading to the ger^ral welfare? Where do the gentlemen get 
this principle?" Larry asks students '^diat do you think about that?" 
One male student, Jim, asserts that once weaken starting working, 
uneirplcyment increased. A second male student agrees with Jim that 
"society ran more smoothly but in the long run having women work 
benefits society. " At this point Carol asks, "Vlho gave men the ri^t 
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to take the pcwer?" Larry ignores her question and asks, "Is there 
any roan here \^o vrcaild stay hcane if his wife could roake a better 
living?" Two roale students respond, one suggesting he'd be bored and 
the other stating he'd quit if he was a secretary or garbage 
collector* 

Larry then reintroduces his central question of the day, "What do 
you think was the nost critical case wcinen mi^t have had for 
equality?" A student misinterprets his question, suggesting that the 
ri^t to vote is most iitportant because "it gives you the opportunity 
to change other things." Rather than rephrase the question Larry 
moves on to another question, "Is there anythir^ in the present time 
that justifies women in terms of their case for equality?" A few 
female students point out that women possess equal brain power, and 
that wcsoen should receive the same pay as a man if doing the same job. 
A third female. Sue, asserts that wcsnen are inferior physically and 
that it's wrong to Icwer the standards for policemen. 

larry then prc^xi^es to the class that "we try to conpose a 
possible amendment that you would put in regarding wornsn's ri^ts." 
Bie following discourse ensues: 

Student: I don't think it's ri^t to have an amenditent. You can't 
e>5)ect this to hajpen over ni^t. It has to be a gradual 
evolution. 

Larry: Why should women have to wait? 

Student: An amsndmsnt is not going to change anything. 

Larry: Why don't the women revolt then? 

Student: If we start giving the women these ri^ts then we have to give 
everyone else these ri^ts. 

larry: How many people are in favor of the Equal Ri^ts Amendment? 
[hand a>unt reveals some females and no males favor it] 
Why would a vranan not be in favor of EE?A? 

Various reasons are then offered by Larry and a few students 
(incliiding fooaales) , such as women may not want to be drafted, their 
hu^^ands may be entitled to alimony in a divorce, etc. The bell 
rings. 

•r 

Reviewing the above lesson witJi respect to the 10 dimensions of 
thou^tfoli^ss, very few positive features emerge. Few topics were 
discussed, specifically the issue of vranen's ri^ts in 1848 and today. 
However, there^ was very little content coherence as discussion junped 
frcan a ccarparison of Declarations, to assessing the merits of the 
Seneca Falls argument, to determining vMch ri^t was most ittportant 
for wcanen, to the reading of a quote, bade to the central question. 
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and finishing with a series of questions regardir^ vrarnsn's ri^ts 
today and the cxanstruction of a nei/ ERA amendment, Uie central 
question was posed on three occasions but within minutes it was lost 
to digression, larcy failed to irodel thou^t fulness, because he did 
not: a) share his reasoning on the various issues addressed — i^xcstpt on 
the central question \fdiere he dismissed the Seneca Falls document on 
the questionable grounds that only wcffnen advanced it; i.e., the 
document was 'Very one--sided; b) conpliitient or encourage students' on 
their tliinldng; c) respond to seriously scxne of the questions raised 
or conments made by students; or, d) require that the student respond 
to a preceding oxnment or question. Students usually did prxsvide 
their reasoning, in part becavise it is a department policy and because 
th^ are hi^ acMeving students. Not once, however, did larry 
request that a student provide additional si^^port for a statement or 
add further embellishment. Because there was nothing irore than a 
"show and tell" of statements and reasons, careful consideration did 
not take place. Since Socratic excihange is often involved in careful 
consideration it is not surprising that it was not also observed. 
Some of the questions posed and issues raised were quite challei^ging, 
but becaiase students were given little time to think before responding 
and because there was no carefta consideration, students were not made 
aware of the difficulty of some questions. Original responses were 
not solicited by the teacher and students v^re not shown that 
disagreement exists among authoritative sources. 

Psychology Lesson of A Hicfo Scorer with Students of Diverse Prior 
Achievandent 

Ihis psychology class lesson is part of a unit on behavioral 
psychology* Hans has two goals for today's lesson: a) enhance student 
understanding of classical conditioning and its principles (i.e., 
generalization, discrimination, and extinction) ; and,"" h) hi^i^t the 
general point or rule about thinking that one must always check and 
questio one's assuitptions. Both goals are pursued by placing students 
in the challer^ing role of behavioral therapist/problem solver. 

Hans begins with a "RAP" (review and preview) of classical 
conditioning. Students are told of a relationship between dog. Bowser, 
and his master, Alphonse. "Whenever Alphonse rips off some paper 
towels and puts on his coat. Bowser kncws it is walk time and barks 
with joy. When Alphonse pulls off paper towels for dinner napkins, 
however, Bcwser does nothing." Students in small groi?)s are to solve 
3 problems using the principles of discrimination and extinction: 
diagram the original classical conditioning that occurred with the 
leash, e3q)lain hew the principle of generalization led Bowser to bark 
with joy vAien observing the coat and paper towel, and explain vAiy 
Bcwser doesn't bark with joy when Alphonse pulls off a paper towel for 
vse as a dinner napkin. 

Following 10 minutes of discussion in small groi^^ the class 
reconvenes. Little time is spent on the first problem, because few 
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Students had difficulty understanding and diagraming Bowser's 
cxjnditioned response of jcy to the conditioned stiinulus of the leash. 
It is diagramed on the board as follows: 



TtiB second problem proves difficult for more students. One 
student correctly esqplains how Bowser, throu^ the principle of 
generalization, connected the leash to Alphones's coat and the ripping 
of paper towel. When a few puzzled lodes persisted, Hans provided 
clarification throu^ questioning rather than direct explanation: 

Hans: 'Without generalization when would the dog bark vdth jcy?" 

Student: "When the leash is on his neck." 

Bans: "That's ri^t. As soon as the leash is on his neck. You could 



even have it two inches in front of ham and without 
generalization what would he do, Ed±Le?" 



Eddie: "Nothing." 

Hans: "Ri^t. So generalization is the essence of all of this 



Following a few additional points of embellishment, HEins directs 
the class to the third problem ("the hard part") , and requests a "good 
bull's-eye ex^ilanation." Ihe first student-volunteer is told that his 
answer is "on target but not a bull's-eye." A few other students offer 
"on target" explanations rathjer than bull's-eyes, so Hans again 
intervenes with questions to elucidate understariaing. Students ccane 
to see that throu^ the principle of discrimination Bowser eaperiences 
jcy only vftien the paper towel is paired with the coat. At this point 
a student asks; "What if the weather changes and you don't need a 
coat?" Hans rewards her with an enthusiastic "Ahhhh", and states that 
he too had thou^t about that possibility. A^in, instead of 
providing an answer Hans asks her vdiat she thinks would happen. When 
she states "releam everything", H^ns and the class lau^ at the 
thou^t of hew confused the dog must be at certain tiroes of the year. 

Having corpleted the RAP, Hans states that for the rercainder of 
the period, "We're going to have some real interestirg stuff go on 
here. It's going to be fun." Using the blacWDoard to outline key 
points traci a reading assignment of the ni^t before, he quickly 
reviews the 3 types of behavioral therapies discussed in the article: 
forming new conditioned responses, using aversive conditioning to 
remove negative behaviors, and applying reciprocal inhibition to 



xmconditioned stimulus (UCS) 



-> unconditioned response (UCR) 
(joy) 



(walking outside) 



conditioned stimulus (CS) — 
(seeing Alphonse with leash) 



•> conditioned response (CR) 
(jcy) 



learning." 
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resmove phobias. He then announces that tcday the class would focus on 
the first of the three behavioral therapies (i.e., forroirg a new 
conditioned response), specifically how the student-therapists will 
help Hans stop his bedwettii>g. Before doirig this, Hans quickly 
reviewed Pavlov's fainous classical conditioning experiment with a 
diagram (i.e., UCS/food — > uoysalivation generalizes to cS/'bell 
— ^> ciysalivation) . [Hans indicated later that this itonentary retuir. 
to Pavlov was designed as a "trap" to ensnare students in their 
thinking about the i?)camii^ bedwetting problem. Hie thinking trap 
that students would now tend to view the bell as only a conditioned 
stimultis. However, as will be seen, in order to correctly diagram the 
bedwetting problem students must define the bell as an unconditioned 
stimulus rather than a corKiitioned stimulus as in the Pavlov diagram. 
Bans then uses this thinking episode to highli^t the general point 
about the iitportance of guestioniiig one's assumptions vdien thinking.] 

Returning to the bedwetting problem, students are told that Hans' 
mattress contains a bell that rings vAienever the mattress beooaiies wet. 
In small grou5>s students are to diagram a conditioning schedule that 
will help Hans with his problem using some of the following tenns: 
bell, full bladder, urination, and wake-i?). Followii>g lively small 
groqo discussions, representatives from two different groups go to the 
board to diagram their group's prescribed conditioning schedule. 
iXiring discussion various students and Hans find problems with both 
schedules. For exanple, one group suggested the followii>g: 

UCS > UCR CS > CR 

(full bladder) (urinate) leads to (bell) (urinate) 

IXiring the analysis Hans laughingly points out that he paid the group 
$500 for a therapeutic intervention that now has him urinatii>g every 
time he hears a bell! '.'You've now created a far bigger problem for 
me." 

Four more groups write their conditioning recommendation on the - 
board, but again class discussion reveals flaws in each. As each 
attenpt fails, Hans provides encouragement to keep students thinking as 
well as comic relief: '*Gee, I hope we can get this or I'm really in 
trouble... I like to see you guys think. It's a tou^ one. Ohere is a 
solution folks, and it's a good one." Finally, Hans must offer a hint. 
He tells the class that he purposely reviewed Pavlov before they got 
started on this problem to put them in a mindset that would make 
solving the problem more difficult, if he hadn't done this, he claims 
50% of the students would have solved it already. Within a minute a 
student provides the correct conditioning relationship, plaining how 
generalization will lead Hans to associate the bell with a full bladder 
Bjnd waking up. 

UCS > UCR CS >CR 

(bell) (wake up) leads to (full bladder) (wake up) 
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Fol?.cwing a few points of clarification, Hans concludes by 
explaining to students the r^eason for the Pavlov review and his desire 
to trap students in their thinking. He states; 

'•Giis is a tou^ one. Some good thinking went on here. I was 
particularly excited when I saw Shane using \Aiat le^^^el of thiriking? He 
started eliminating these [i.e., the terms] by saying there are no 
vmconditianed stlitiulus and responses left. But the^re was one left 
wasn't there, bell vs wake vp. Shane were you kind of cai^^t in the 
Pavlov trap? See v*iy I set that \jp7 Just a little point about 
thinking here. What's one thinking error that we oftei run into? We 
don't question our assunptions, ri^t? What was our assunption there? 
Ihat bells are always a conditioned stimulus...", 

Bie lesson ends with Hans previewing the reciprocal inhibition 
hcanework assignment for tcffioinxw, one based on a Gary Larson cartoon in 
whidi students are to make various evaluations of "Professor 
Galla^ier's controversial technique of simultanjiously confronting the 
fear of heists, snakes, and the dark." (See worksheet below). As 
students are scrambling out the door, Hans makes a final attatpt to 
instill motivation and a good effort; "It's going to be tou^. It's a 
good one. Do a good job on it. I want good work. . .Good thirikii^ today, 
Shane..." 

Reviewing the above lesson with respect to the oI:^ervational 
scales, many outstanding features emerge. Few topics were discussed, 
specifically classical conditioning with respect to Bowser and to Hans' 
bedwedding. The content was presented and discussed in a very coherent 
manner, and the teacher modeled thoughtfulness in his own thinking and 
in his appreciation and enthusiasm for the thinking of students. The 
problems posed were challenging, and students' responses were carefully 
analyzed by the teacher and students. Students were given time to 
think during small grocp, and Socratic exchanges occurred on occasion 
between Hans and students. The lesson did not reveal to students 
disagreement among authoritative sources, thoo^ later in the week 
students critiqued behavioral therapies from previously studied 
Freudian and Jungian perspectives. 

Civics lesson of a Lower scorer with Students of Low Prior 
Adiievement 

This lesson focused on the ^stem of checks and balances between 
tlie executive, judicial, and legislative branches of the United States 
federal government. Lloyd began by reviewii^ some of the points 
covered in the previous day's class discussion. He asked \^y the 
Constitutional Convention wanted to divide power. A stijdent responded, 
"The government might destroy freedom." Lloyd acknowledged the comment 
as correct and reiterated the point; "Good, there was a fear they would 
not have enou^ freedom." Lloyd then asked students to name the 
document of 1776, and to name the first plan of government. Students 
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qinckly supplied the correct answers; i.e., the Declaration of 
In(5^)end£nce, ar^ the Articles of Confederation. Uqyd rendnded the 
class that "in 1787 we went to a federalist system", and then asked 
them to €3§)laiji this syf^tem. Failing to get the desired response, he 
stated that the 3 branches of the federal government were set xjp due 
to "the fear the peqple had about the national government beccaning too 
powerful." Students si^plied the nan^ of the three branches on 
Lloyd's request and their respective function (i.e., 
legislative — "make laws" , executive— "enforce laws" , and 
judicial~"interpret laws") . with the review of yesterxJay's 
discussion corryoleted, lioi'd told them this tcpic is inportant as it 
:rauld appear on tha final exam much later in the school year. 

Before discussing the system of checks and balances, Ilqyd in a 
few sentences summarized how the Articles of Confederation proved 
inadequate, and how the framers wanted a stronger central goverrmvent 
but^ one that could be controlled. He then asked students to open 
their textbook to a paragraph that summarized how the three branches of 
government checked and balanced the power of each other. A student was 
asked to begin reading the paragraph; "Ihe executive branch checks the 
legislative branch by signing or vetoing a bill. " Iloyd then 
intem^rtzs, asking vSiat the word veto means. Once this word was 
clarified the student resumed reading to the class, explainirrj how 
Congress can override the president's veto with a 2/3rds vote. Uqyd 
at this point asked students to take time to write in their notes "vrtiy 
we have a system of checks and balances." While students wrote Uqyd 
read aloud the text's answer to this question; "Ihe purpose of the 
checks and balances is to make sure that no branch of this national 
government can become too powerful." 

Discussion resumed with Ilqyd asking students how the president 
checks the Si:5)reme Court. A student correctly responSed that the 
president appoints members to the Siprerte Court, without clarifying 
hew the president's power of appointment in fact checks the power of 
the Sipreme Court, Uqyd directed students to the apprqpriate line of 
their worksheet where they were to write down "President appoints 
members of the Sipreme Court." Uqyd then asked students how often the 
Si^reme Court checks the President. Follwii^ a few incorrect 
responses by students, he e>^lained that the Si:5)reme Court can tell the 
president^ to enforce certain laws and tell him \rtiich laws are 
constitutional and unconstitutional. Uqyd reminded students that the 
president cannot just go out and maJce a law; "Congress makes the law. 
Die Sipreme Court interprets the meaning of the law and vAiether it's 
constitutional or unconstitutional. He has checks. He's not an 
enperor or dictator^" 

At this point students are asked to finish the worksheet on the 
system of checks and balances at their desk. Uqyd informs them he 
will yyujjz around and lend assistance, lo minutes later st»adents had 
coDpieted the worksheet and Uqyd resumed discussion, students were 
askfed hw Congress checks the president . A student correctlv 
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responded, , "It can pass laws over the president's veto." Two itore 
questions are asked (i.e.. Hew does the Sa^ istB Court check CJongress? 
and, How does Congress check the Sipreitve Court), with students 
providing the desired answer before the bell rings ending class. 

Reviewing the above lesson with respect to the 10 diioensions of 
thcw^tfulJiess, a few positive features emerge. Few tqpics were 
discussed, specifically the separation of pcwer aitong the three 
branches of government and the system of checks and balances . 
Althou^ the questions posed required only sinple recall, students \^re 
given sufficient tiitve to think before responding. Finally, the 
content, though covered very siqperficially, was for the most part 
coherent. 

The lesson failed, however, with respect to the other dimensions 
of thm^tfulness. The tasks posed ty the teacher were not 
challenging, rather students recited frm the textbook responses 
desired by Ilqyd. Ihis, in part, led to little modeling of 
thcw^tfulness, as few situations arose vdiere the teacher had to think 
throu^ a question or problem with students. Ilqyd did not encourage 
or reward students* thinking, nor did he e>dubit much enthusiasm or 
appreciation for the topic and ideas being discussed. Students did not 
offer reasons for their answers, but instead only siqpplied phrases from 
the textbook or the previous day's discussion. Without challenging 
tasks or reason giving, it is not surprising that careful consideration 
or Socratic exxiianges failed to take place. Finally, original 
responses were not solicited by the teacher, and students were not 
shewn that disagreement exists among authoritative sources. 

OOMEARIWS HIGH AND LOWER SOCBRERS CW INSITOCriaHAL FORMflT, REftDIMG 
MftTERIAIS, fiND WRITING TASKS : 

As a final point of ccarparison the frequencies with \Mch hi^ and 
lower scorers engage students in various instructional formats, and 
types of reading materials and writing tasks are presented. 

Beginning with instructional format, part of virtually every hi^ 
scorer's lesson contains a vAiole group teacher centered discussion, 
ccnpared to 2 of every 3 lessons among IcMer scorers. Rouc^y 1 in 4 
lessons of hi^ scorers involve students in small groc^) work v^ereas 
small group work is virtually non-existent in the lessons of lower 
scorers. Ihe reverse situation is found with respect to lecture; that 
is, 1 in 4 lessons among Icwer scorers include lecture cotpared to 
virtually no lecturing among high scorers. Similar findings en^e 
with respect to recitation, as 1 in 1 lessons ejdiibited a recitation 
format among lower scorers o^mpared to no recitation anr^ng hi^ 
scorers. In neither the classes of hi^ or lower scorir^ te:ichers was 
there much studen^- centered discussion, viewing of filins or 
filmstrips, seatwork, or oral reports (see Table 8) . 
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Turning to ti^(s) of reading materials used during lessons, on 
average, Icwer scorers* use a textboQk(s) 1 of every 2 lessons, 
corpared to approximately 1 in 5 lessons amcang hi^ scorers (see Table 
9) . One in 3 lessons finds both groups vising primary source 
materials, tut rarely does either grxxjp vise articles, literature such 
as novels, poems, etc. One in 3 lessons finds hi^ scorers cccpared to 
1 in 6 lessons amcang Icwer scorers using "other" reading materials. 
For the most part these are reading materials the teacher has 
developed using a photocopier or mijoeograph machine. Finally, hi^ 
scorers do not use any reading materials in about one-fourth of their 
lessons, vjhereas lower scorers do not use any readir^ in about 
one-tenth of their lessons. Ihis may be due to hi^ scorers' greater 
ernphasis on teacher-centered whole grocp and student-centered small 
grtxp discussion, cccbined vdth Icwer scorers' greater enphasis on 
recitation and seatwork activities; 

For the most part both groups require seme form of writing during 
class. ^ Neither gixxsp is likely to assign paragraphs or essays, 
research papers, or "other" type^ -^f writii^ tasks (see Table 10) . Die 
most frequently assigned type of writii^ among both hi^ and lower 
scorers is that of notetaldng. Ihree of 4 hi^ scorers* lessons 
require notetaking ccitpared to 1 in 2 lessons amoixf lower scorers. 
Hi^ scorers are more likely than lower scorers to require writing 
ccfftplete sentences, but both groips require this infrequently. 
Worksheet or slaort answer writing tasks are equally stressed, about 1 
in 3 lessons for both groaps. 

Ihe instructional formats , classroan reading materials , and 
classroom writing tasks of hi^ vs lower scorers can be summarized as 
follows. Hi^ scorers' lessons almost always include teacher-centered 
v4iole gnxp discussion, with an occasional small gro^ activity. One 
is likely to find hi^ scorers using primary sources and 
teacher-generated reading materials rather than textbooks, students 
of hi^ scorers are li)cely to be found taOdiYj notes, and occasionally 
ccopleting worksheets and other short answer writir^ tasks. lower 
scorers* ler yns are also likely to include teacher^<:entered v*iole 
groi^ discussion. However, unlike hi^ scorers, one is also likely to 
observe during these lessons a lecture or recitation instructional 
format. Textbocdcs are the main reading staple, but like hi^ scorers 
primary sources materials are also frequently used (34%). Notetaking 
is slic^itly less enjlxasized by lower scorers but canman to a majority 
of lessons, vAiile work^eet/short answer writing assignments are 
required less often but at a rate corparabie to hi^ scorers. 

OOWCIDSIOW 



^ m only 5% of hi^ scorers' classes and 16% of lower scorers* 
classes was no form of writing required. 
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Hi^ scorers cxarpared to lower scorers consistently display 
si5)erior performance on virtually every dimension of thou^tful 
practice* Differences are sli^tly less pronounced when teachers* best 
lessons are caipared, yet even here sizeable differences einerge on 6 of 
10 dinaensions* Across all lessons sizeable differeix^ emerge on 7 of 
10 dlinensions, 6 of vdiich exceed 1.5 standard deviations (i.e., 
substantive coherence, Socratic dialogue, carefUl consideration, few 
tcpics, challenging tasks, and teacher nodeling thou^tfulness) . 

Differences between the two groins emerge on ether elenents of 
practice. Hi^ scorers engage students in teacher-centered vdiola group 
discussion almost every lesson coarpared to only a majority of lessons 
for lower scorers. Lecture and recitation occur in approximately 
one-fcurth of lower scorers' lessons whereas these two instructional 
formats are generally not found in the lessons of high scorers. As 
mi^t be expected, small group discussion ocoars icore frequently in 
hi^ scorers' lessons (one-fourth of the time), but lower scorers 
rarely enploy this format. The many sizeable differences found 
between hi^ and Icwer scorers on the 10 dimensions of practice may, 
in part, be attributed to the use of lecture and recitation fontats 
which are less conducive to facilita'ting thou^tful discourse than 
vdiole grcxp or small group discussion. 

In li^t of the disparagii>g assessment textbooks have received 
concerning the facilitation of students* thinking, it is interestir^ to 
note that lower scorers use textbooks in about one-half of their 
lessons \*dle hic^i scorers use textbooks in less than one-fifth of 
their lessons. Hi^ scorers on the other hand are twice 33 likely to 
provide students \d.th teacher generated readily materials. This may 
e:?)lain, in part, less content coverage (i.e., few topics) by hi^ 
scorers as the reading materials losed are generally less survey 
oriented than standard textbooks. 
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TABLE 1 

CIASSKXM EE»CIICE OMEaRISCNS iiOOS AH, IES9CKS (n=88) 



(n=44) 
Hi^ Scorers 


(n=44) 
Lower Scorers 


5?l-:i-2?2 
o.u. 

pooled 


Mean^ 


S.D. 


Mean5 


S.D. 


upgre ^- ,>r 
Dirfevenca 


4.55 
4.3C 
4.23 
4.57 


( .73) 
( .73) 
( .64) 
( .70) 


3.59 
2.66 
4.20 
3.36 


(1.04) 
( .75) 
( .70) 
(1.12) 


1.74* 
1.84* 

.06 
1.91* 


4.55 
3.93 
4.35 


( .63) 
( .79) 
( .45) 


2.80 
2.82 
3.24 


( .93) 
(1.26) 
( .58) 


1.84* 
1.28* 
1.57* 


4.09 
2.84 


(1.01) 
(1.24) 


2.34 
1.52 


( .94) 
( .88) 


1.68* 
1.69* 


2.18 


( -95) 


1.68 


(1.01) 


.81* 


1.95 


(1.33) 


1.70 


(1.00) 


.35 



5CAIE 

-few topics 

-substantive coherence 
-time to think 
-challenging tasks 
-teacher models thou^t- 

fulness 
-students give reasons 
-BOOAV (above 6 scales) 

-careful consideration 
-socratic dialogue 
-teacher invites original 

responses 
-disagreement axnong 

authoritative sources 



* denotes a statistically significant difference between the two 
grtxqps at the .05 level- 
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TABLE 2 



CLftSSKXM IKacnCE C3CMEM?IS0MS CN SEEBCT (BEST) lESSOB (n=20) 



scaiE 

-few topics 
-substantive oSiereiice 
-tiioe to think 
-chedlenging tasks 
-teacher laodels thou^t- 

fUlness 
-students give reasons 
-HOGffiV (above 6 scales) 

-careful consideration 
-socratic dialogue 
-teacher invites original 

responses 
"disagreement among 
authoritative sources 



(n=13) 



(n=7) 



Hi^ Scorers 


Lower Scxsrers 












Degree of 


Mean 


S.D. 


Mean 


S.D. 


Difference 








( .93} 


1.16* 


4.69 


( .48) 


3.00 


( .00) 




5.00 


( .00) 


4.29 


( .49) 


1.83* 


A.ll 


( .44) 


4.43 


( .58) 


.54 


4.85 


( .38) 


3.86 


( .69) 


1.04* 


4.38 


( .51) 


4.14 


( .69) 


.28 


A.ll 


( .16) 


4.00 


( .24) 


1.08* 


4.69 


( .48) 


3.00 


( .58) 


1.82* 


3.38 


(1.39) 


2.00 


(1.16) 


1.78* 


1.92 


( .64) 


2.29 


( .95) 


-.58 


1.46 


(1.13) 


2.57 


(1.62) 


-1.56* 



* denotes a statistically significant difference between the two 
groins at the .05 level. 
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CIASSRDCM BRacnCE QMEKRISCIB CK NCWr-SEIECT (lOatRINING) lE^CIS (n=€8) 

(n=31) 

Hie 

SCftlE 



-few topics 

-substantive coherence 
-time to think 
-challenging tcisks 
-teadier nodels tiicught- 

fulness 
-students give reasons 
-HCXCAV (above 6 scales) 

* careful consideration 
-socratic dialogue 
-teacher invites original 

responses 
-disagreement anaang 

authoritative sources 



Hi^ Scorers 
rfean S.D. 


Icwer Scorers 
Mean S.D. 


Degree of 
Difference 


4.39 


( .80) 


3.46 


(1.02) 


1.69* 


4.13 


( .76) 


2.59 


( .80) 


1.72* 


3.90 


( .47) 


4.19 


( .74) 


-.73* 


4.48 


( .77) 


3.16 


(1.09) 


2.10* 


4.42 


( .67) 


2.59 


( .83) 


1.92* 


3.74 


( .82) 


2.57 


(1.19) 


1.35* 


4.18 


( .41) 


3.09 


( .51) 


1.53* 


3.84 


(1.07) 


2.22 


( .95) 


1.74* 


2.61 


(1.12) 


1.43 


( .80) 


1.51* 


2.29 


(1.04) 


1.57 


( .99) 


1.17* 


2.16 


(1.37) 


1.54 


( .77) 


.87* 



* denotes a statistically significant difference between the two 
gnx^js at the .05 level. 
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TABLE 4 

SDMCVBY OF TaHDES 1-3* 

(Table 1) (Table 2) (Table 3) 

SCALE ALL UESSOiifS BEST TESSOHS REMAINING I£SSC»JS 

-few topics XXX 

-substantive cdierence XXX 

-time to think X 

-challenging taslcs X X 

-teacher models thou^t- XXX 
fulness 

-students give reasons X X 

-HOTAV XXX 

-careftiL consideration XXX 

-socratic dialogue XXX 

-teacher invites original x 
responses 

-disagreement among -X 
authoritative sources 

* An indicates that the degree of difference between hi^ and 
Icwer scoring teachers esoceeds 1 standard deviation in favor of hi^ 
scorerSf vMle a "-X" indicates that the degree of difference exceeds 1 
standard deviation in favor '^f lower scoirers. 
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TPkVSE 5 

isBQDENCY msmmnross of high mi ioker scosebs as the 

10 niMEUSICNS OF BCOaHSUL CLA^RDCM HRSCnCE 



High Scorers lower Soorers 





1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


-Few Topics 


0 


1 


3 


11 


29 


1 


5 


15 


13 


10 


Odierence 


0 


0 


7 






JL 






o 


u 




0 


0 


R 






o 

U 


JL 


A 
H 






-Challenging 
Tcisks 


0 


1 


2 


12 


29 


1 


11 


11 


13 


8 


-Teacher Models 
Ihcajj^tfulness 


0 


0 


3 


14 


27 


1 


19 


14 


8 


2 


-Students Give 
Keasons 


0 


2 


9 


23 


10 


5 


20 


1 


14 


4 


-Careflil 
Consideration 


1 


3 


5 


17 


18 


7 


20 


14 


1 


2 


-Socratie 
Dialogue 


6 


13 


13 


6 


6 


2S 


12 


2 


1 


1 


-Original 
Eesponses 


10 


21 


9 


3 


1 


24 


15 


2 


1 


2 


-Authorities 
Disagree 


25 


6 


7 


2 


4 


26 


8 


8 


1 


1 
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TaSTiS 6 

CEMEaRISCN OF HTGH SOSERS' BEST lESSONS WITH OHEIR EBKAINING lESSCNS 





(n=13) 




Best 


SCAIE 


Lessons 


-few topics 


4.92 


-substantive cxJierence 


4.69 


-tia^ to think 


5.00 


-diEdlenging tasks 


4.77 


-teacher inocJels thou^t- 




fulness 


4.85 


-students give reasons 


4.38 


HOflAV (the above 6 scales) 


4.77 


-careful consideration 


4.69 


-socratic dialogue 


3.38 


-teacJier invites original 




responses 


1.92 


-reveal disagreement ainong 




authoritative sources 


1.46 



(n=31) 

Remining (n=44) Degree of 



Lessons S.D. Difference 



4.39 ( .73) .72 

4.13 ( .73) .76 

3.90 ( .64) 1.71 

4.48 ( .-70) .42 

4.42 ( .63) .69 

3.74 ( .79) .81 

4.18 ( .45) 1.32 

3.84 (1.01) .84 

2.61 (1.24) .62 

2.29 ( .95) -.39 

2.16 (1.33) -.53 



IV - 35 

TftBTE 7 



COffiARISCN OF ICHER SOXSSS* BEST lESSOMS WTIH THEIR RaSKSnSG I£SS0N5 







(n=37) 








Best 


Renaining 


(n=44) 


Dsocee of 


scaiE 


Lessons 


Lesscais 


S.D. 


Difference 


—few "topics 


4.29 


3.45 


(1.04) 


.80 


-substantive cxiierence 


3.00 


2.59 


( '75) 


.55 


-time to think 


4.29 


4.19 


( .70) 


.14 


-diallenging tasks 


3.00 


3.16 


(1.12) 


-.14 


-teacher mocJels thou^t- 










fulness 


3.86 


2.59 


( .ss) 


1.37 


-students give reasons 


4.14 


2.57 


(1.26) 


1.24 


HOdAV (the above 6 scales) 


4.00 


3.09 


( '58) 


1.57 


-careful consideration 


3.00 


2.22 


( •94) . 


.83 


-socrat5.c dialogue 


2.00 


1.43 


( •88) 


.57 


-teacher invites original 








responses 


2.29 


1.57 


(1.01) 


.72 


-reveal disagreement among 










atithoritative sources 


2.57 


1.54 


(1.00) 


1.03 



TABOUE 8 

OCMEaRESCN OF CLaSSRDCM INSmJCTICNaL FUKMai (n=88) 



Forxnat 

-teacher centered discussion 
-student small groips 
-student centered discussion 
-lecture 
-film 

-recitation 
-seat work 
-oral r^)orts 



Hi^ Scorers lower Scorers 

% of lessons % of lessons 



96 


66* 


23 


2* 


2 


0 


0 


27 


2 


0 


0 


23 


0 


5 


2 


2 



* denotes a statistically significant difference between the two 
grcu^ at the .05 level. 
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TftBTE 9 

OCHBJVRISCN OF CEASSRDCH KEM}INS HKCERIflUS (n=88) 

Hi^ Scorers Lower Scorers 

Materials % of lessons % of lessons 

-text 18 52* 

-primary source 30 34 

-article 7 6 

-literature o 2 

-other 36 16* 

-none 23 9 

* denotes a statistically significant difference between the two 
groi?)s at the ,05 level. 
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TftHTf! 10 

OWPARISCN OF CIASSRDOM KECTEINS TASKS (n=«8) 

Hi^ Scorers Ixwer Scorers 

Format % of lessons % of lessons 



-outline 


0 


0 


-class notes 


73 


55 


-worksheets/short answer 


30 


34 


-ccnplete sentences 


21 


9 


-paragrs^ or essay 


0 


7 


-research paper 


0 


0 


-other 


5 


0 


-none 


5 


16 



* denotes a statistically significant difference between the two 
groi:ps at the .05 level. 
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SiUDbMT EEESEECEEVES ON 

cx^asanrns (SKOJEtisim oirrigoiim 
Bobert B. Stevenson 

Much research in hi^ schcx^ls indicates that students are rarely 
challenged to use their ininds (Cusick, 1973; Goodlad, 1984; Pcwell, 
Eeanrar, & Cohen, 1985; Sizer, 1984) . Ifer^ teachers claim that a major 
reason for this situation is students' lack of interest in or passive 
resistance to tasks that challenge them to think and their desire to 
invest only a ininiitial aitount of effort in academic work (Haitipel, 1986; 
Tye, 1985) . Biis contention seems to be si^^ported by reseairh 
indicating that in nany hi^ school classrooams students' covertly 
negogiate an agreement to exchange their ccatpliant labor for minimal 
academic demands from their teacher (Pcwell, Farrar, & Cohen, 1985; 
McNeil, 1983, 1984; Sedlak, Wheeler, Pullin, & Cusick, 1986). Others 
(Goodlad, 1984; Sizer, 1984), however, suggest that students are 
indifferent to schoolwork because they are not intellectually 
challenged, fcait instead ,are confronted with a curriculum which 
enphasizes lower order cognitive tasks (i.e., the memorization of 
mindless infonnation and the mechanistic application of algorithms) . 

One way to address this issue is to examine the perceptions of 
students regarding the kinds of academic vjork that make them think 
hard. This paper presents the findings oi: an enpirical study vMdi 
inquired into the characteristics of cdirses, lessons, and tasks that 
students report to be co; litively ca>alienging, and their reactions to 
such challenges. 

MgnPDQIOGY : 

Ihe data for findings in Chapters V and VI were collected as 
e)?)lair^ in the Introduction. A total of 45 students, nine from each 
of the five geographically diverse hi^ schools, \^re interviewed for a 
total of approximately three hours on three different occasions spread 
over the 1986-87 school year. Interviews were tape recorded an3 
students also were administered two questionnaires. Ihree students were 
selected frcm each of three different social studies classes in each 
sdiool by the teacher on the basis of being representative of the top, 
middle and bottcm third in the academic achievement distribution of the 
class. Hie ccnposition of the sairple of students was 47% male, 53% 
female, 24% hi^x school freshmen, 29% sqphcarores, 31% juniors, aM 16% 
seniors. By race, 73% of the sample ware White; 22% were Black, and 4% 
were Asian. 

Berikuse of logistical constraints, the sairple size is relatively 
small. A larger sanple would have reduced the opportunity for in-deptJ*! 
interviewing needed in an e^qploratory stud/. Hcwever, since the 
students represented a cross-section with respect to academic 
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achieverosnt (as boma out by the distribution of grade point averages) , 
and the schools were located in diverse ocaiiinunitY contexts, the 
possibility of generalization should not be ruled out. Since both 
teachers and students were aware that the general purpose of the 
research was to examine HOT in social studies, it is possible student 
responses were biased toward favorable reactions to this subject. 
Hcwever, raaxt^ students neither praised social studies nor their social 
stidies teacher, and nearly all were able to give consistent reasons 
vrtiy certain classes and activities beyond the observed social studies 
class were challenging. In fact the consistency of student responses 
across a nuniber of different learning contexts, at three different 
points during the school year, suggests the lack of a systitatic bias 
toward the dbserved social studies class. 

In order to identify vtot students consider to be "challer^ing" 
classes and tasks, they were questioned about learning in several 
different contexts. Specifically they were asked to describe: (a) the 
most challer^ing course or subject th^ had taken both in the current 
school year and throu^out hi^ school and the reasons \^y it was 
challenging, (b) exairples of challenging activities or tasks in their 
present social studies course that was observed by a HOT project 
researcher and their reactions (i.e., frustrations and satisfactions) 
to such activities, (c) specific questions or tasks from three social 
studies lessons observed by a researcher vMch made them think hard and 
the frrastrations and satisfactions they experienced from thinking about 
those questions or tasks, (d) the t!/jpes of lesson formats that 
generally challenge them to think hard, and (e) types of writing tasks 
that generally challenge them to think hard. 

Ihree coding schemes were developed for categorizing students' 
verbal responses to the cpen-ended int^iew questions (and to follow- 
probes by the researchers). Pilot schemes were constructed by 
generating categories based on relevant typologies suggested in the 
literature and the author • s personal knowledge . Each scheme was 
reviewed by two dther researchers fxm the HOT project aixi then tested 
on 10 randomly selected responses. After necessary revisions, the five 
schemes were subjected to a reliability analysis losirg a different set 
of 10 responses and two volunteer graduate students vAio were trained in 
coding the data. After reliabilities had been coffiputed the raters 
(including the author) arrived at a consensus on the coding of disputed 
responses and the need for further revisions to some of the schemes. 
Using the revised schemes the author then coded the remaining 35 
responses to each question. 

EENDINGS ; 

The Most Ooanitively Challencrmcr Subjects 

Over one third of the sample (see Tables 5A and 5B) rated a social 
studies cairse the most challenging class they had taken during the 
1986-87 school year, and the most challenging they had taken throu^out 
their hi^ school career. Ihe majority (i.e., 71%) of the social 
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studies cxwrses mentioned as the most challenging class last year were 
history (compared with 60% of history courses in the project^s sanple) , 
with the other five classes coaiprising two general social studies, two 
eccaioKdcs and one psychology. Ihe proportion of social stidies courses 
cited reprttisents almost as many as the ccnibined total for science and 
mathematics vMch are traditionally regarded as the tou^ subjects. 
Given the nuniber of subject areas that these students have taken in 
hi^ school (i.e. , between six and ten) , the e35>ect::ed prx:bability (as a 
percentage) of social studies (or any one subject area) being rated the 
most challenging is between 10 and 16.7 percent. 

It should be pointed out, however, that the five schools frcsn 
\diich the subjects for this study were ctotained were selected on the 
basis of having social studies departments whidi errphasized the 
develcpnent of hi^er order thinking. And in four of these five schools 
the principal rated social studies tlie strongest academic department in 
their school. Consequently, this finding of social studies beir^ rated 
the Host challenging subject cannot be considered representative of the 
general population of secondary school students. Nevertheless, it shows 
that in these schools many students fird social studies particularly 
diallerging. 

Table 5A: Subject of Most Oiallenging Class Taken in Present 

Scixxd. Year 
(n=43) 

Rartc Subject Frequency Percentage 



1 Social Studies 17 35.4 

2 Science lo 20.8 

3 Mathematics 9 is. 8 

4 English 8 16.7 

5 Foreign Language 2 4»2 

6 Industrial Arts l 2.I 
6 Cocpiter Science _1 2.I 

48* 100.0 



* Five students nominated two classes as equally challei^ir^ 

Table 5B: Subject of Most Challengii^ Class Taken in Hi^ School 

(n=42) 



Rank Subject Frequency Percentage 



1 


Social Studies 


15 


35.7 


2 


Science 


9 


21.4 


3 


Mathematics 


8 


19.0 


4 


English 


7 


16.7 


5 


Industrial Arts 


2 


4.8 


6 


Foreign language 


1 
42 


2.4 
100.0 
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When asked : '*What was it about this class that made you have to 
think hard?", half the students gave explanations that were related to 
csognitive processing demands (see T&ble 5C) . The irost jErequent response 
can be classified as the demand for workiiig oui: the answer or solution 
to \vhat Simon (1973) has termed a 'H^l-stmcfcured" problem vAiere there 
, is clear defiiiition and goals and a ri^t answer or solution. For 
exanple, one student cotpared working on trigonosaetry problems to a 
puzzle or game in vdiicih she gets satisfaction frcjn workingr it out. On 
the other hand, other students reported that working on an "ill- 
structured" problem, where they had to formulate a response to or 
opinion on a questicai or issue in vMch the answer is prcblematic (and 
vAiere the definition of the prcblem and the rules of solxition 'are 
usually ambiguous), accomted for the reason a course was the iiost 
challenging. In the words of one student: "Ihere were a lot of 
different ways to view the questions. Not direct ('yes- or 'no') 
questions, but rather ^yA)at if. Usually there's more thrji one answer. 
You really had to think it throu^ thorou^y to get to the best 
answer." 

Interestingly, these two different types of problem were 
associated with distinctly different school subjects. Well-defined 
problems were the reason offered to e>^lain the cognitive challer^e of 
classes in mathematics, science, and in one case, auto mechanics. Ill- 
defined problems were mentioned only in reference to social studies. 
Ihese differences were captured by one student vdio described chemistry 
and social studies as equally diallenging but for different reasons: 

"Social studies is different from chemistry. Chemistry involves 
theories, formulas, and a lot of logic. TJiere^'s only one or a 
few final answers. But it's a different kind of thinkii^ than 
social studies. Social studies involves emotions and formirg 
your qpinion is different than thinking v4iy something works. In 
social studies there is no one set answer, but a lot of factors 
involved in defending different solutions." 
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oonprehen- 


7 


16.3 


• 

sion 


5 


11.6 


27.9% 


2 


4.7 


vork & 


8 


18.6 


aremory 


4 


9.3 


losd 


3 


7.0 


25 


1 


2.3 




1 


2.3 


cxjgmtive 


10 


23.2 


processing 






demands 


5 


11.6 


51.2% 


5 


11.6 




3 


7.0 




2 


4.7 




1 


2.3 


teacher 


2 


4.7 


behaviors 








2 


4.7 


9.3% 






peer 


3 


7.0 


behaviors 


1 


2.3 



9.3% 



Table 5C 
Reasons Class is Challencrim 
(nF=43) 

no,* ^ 



inherent difficulty of concqpts 
St cent's lack of backgrcaind in or 
Icn'^vledge of subject 
difficulty of reading material 

quantity of work demanded 
demand for itemorizing present material 
demand for accessing previous course 
material 

demarxi for independent work 
demand for concentration owing to 
uninteresting subject or material 

working out the answer or soltttion to 

well-defined problem 
deciding own position on question or 

issue with prc±)lematic answer 
making inferences 
applying multiple concepts or 

integrating multiple factors 
developing ejqplanations, reasons and 

justifications for ans^wers or opinions 
creating an original idea 

lack of clear guidance in understaj^iing 
naterial or defining task requirements 

demanding e^^jectations and gradiixf 
critieria 

ccarpetitive environmant amcang students 
stimulating contritaJtions f rem students 



* Nunbers add to more than 43 and percentages ar^ to more than 100 as 
more than one category of reasons could apply to einy response. 
Percentages were coatputed by dividing the number of students respondii^ 
in each category by the total number of students (i.e. ; 43) . 



ERIC 



V - 6 



Another student perceived the saiue distirK±ion between mth and 
social studies, but found that the relativistic conception of knowledge 
conveyed in her social studies class mde it itruch easier: 

"Social studies becomes routine after a vMle* You're expected 
to take a side and share your opinions* You can usually think 

of reasons to give and justij^ yourself Calculus has ri^t 

and wrong answers, but in social studies there is no ri^t or 
wrong, usually* In social studies if you raise your hand and 
disagree with somebody it's okay*" 

When asked hew she knows there aren't ri^t answers in social 
studies, this student replied that her teacher told the class there are 
no ri^t or wrong ansv^rs on opinion questions. 

As Table 5D illxjstrates, solving ill-structured and well- 
structured problems were the m>st common reasons students gave for v4iy 
social studies and other subjects respectively made them think hard. 
Alinost one third of the students who reported social studies to be 
their most dhallengiiig class during the past year attributed the reason 
to their being required to decide their position or opinion on a 
question, problem or issue whidi has pxblematic answers. Two of these 
students specifically added that their teachers also demanded that they 
defend and si^port their opinions. One freshman student, for exairple, 
found her history class hard because "yru have to defend yairself on 
every answer you give." On the other hand, one third of the students 
\*o found other subjects made them think the hardest were challenged by 
the task of figuring out the ric^t answer or solution to a well-defined 
pr^lem. 

Table 5D: Most Oooincai Keasois for Ifcminating Social Studies 
and Other Subjects as Most Challenging Class 
in Bast Year 

Rank Social Studies Fo^ All Qthei Subjects fo* %* 

1 deciding own opinion 5 29 solving well-defined prc4>s 10 32 

2 making inferences 4 24 conceptually difficult 7 23 

3 quantity of work 3 18 quantity of work 5 16 

* Number and percentage of students 

Seme students reported that having to make inferences mde a class 
challenging. All but one of these responses referred to a social 
studies class. Ohese four students described how they were required in 
their social studies class "to go beyond the facts" or "to make 
inferences", by discussing, for exairple, the implications of particular 
decisions or events, or possible alternative scenarios and their 
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consequences. For exairple, one student stated that there are itore 
underlying things he (the teacher) wants us to get. In earlier social 
studies classes "you'd read it over, in 1776 they signed the 
Declaration of Independence and we became a free nation, but now you 
have to think about it ... by going bsyond the facts. " 

The same number of responses described the challenge of either 
applying multiple concepts/factors to a question or prcblem (e.g. , "to 
do proofs, _ . .there are so mary different possibilities to consider") , 
or _ developing eaqolanations, reasons aixi justifications for answers or 
opinions (e.g., "she [the teacher] asks you v4iy? She'll challenge you 
further") . In each, of these two categories each, reponse applied to a 
different subject. The exanple frcsn the student who distinguished 
between the different kinds of thinking in chemistry and social studies 
also_ illustrates the challenge in social studies of integrating 
multiple factors given the variety of ways of viewing many social and 
historical questions. 

m addition to demands for hi^er order cognitive processing, a 
significant minority reported that difficulties with conprehension made 
them think hard. Ihis was mainly due to the ccstplesdly or abstractness 
of the conc^jts in a particular subject or their lack of background in 
or knowledge of the subject matter. Ihis raises a second interesting 
distinction between social studies and other subjects. When students 
were asked vAiy a particular school subject was liie most challenging, 
the second most common reason offered was that i±& conceots involved 
are difficult to understand (see Table 5D) . This .ceason w^ roost often 
cited in the case of mathematics and science, especially physics, but 
never in social studies. Students don't seem to fii>d the concepts and 
generalizations in social studies very difficult in themselves, 
althoucii two students mentioned a problem in history of understanding 
events that occurred in a different tiire period to vdiat they have 
ej^jerienced. Social jdies apparently only becomes difficult \*en the 
teacher makes them work or intellectually play with the material. It is 
also possible that difficult or abstract concepts in the humanities can 
be connected more readily to concrete human events and e5$)eriences than 
conospts in science and mathematics. 

These student perceptions seem to be consistent with the lack of a 
research tradition in the social studies field on how to teach and hw; 
students learn conceptually difficult material. Instead, the primary 
focus of curriculum theory and research in social studies historically 
has been the selection of content (i.e., identifyirg vdiat to teach), 
vMch, seems to be more problematic than in other subject fields. 

In contrast to the demands for hi^er order thinking discussed 
above, many students indicated that the work and memory load in a class 
made it challenging, including the quantity of work covered, the amount 
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to be xQeraorized, or the previous course material vfliich had to be 
recalled. Ihe quantity of work demanded was the third itost cormon 
reason loentioned for classes in both social studies and other subjects 
(see Table 5D) . Ihree students found the quantity of work covered in 
their history course very demanding, and five students said the sane 
for either English or, in one case, Spanish, suggesting that the aitount 
of material covered in these subjects is a challei^e for many students. 

Only a few students (see Table 5C) indicated that the academic 
environment created by their peers or specific characteristics of their 
teacher e5?)lained '^y a class was most challenginge In contrast, mary 
students reported that the reason for a particular class being engaging 
was attrihutable to various characteristics of their teacher. 

HiB Challenge of Hicftier Order Ihinkincr . Several fii^iings indicate 
that students find classes that attenpt to prxstcte hi^er order 
thinking challenging. Across all five schools, among students \*o 
ncsodnated a social studies course as the most challenging in (a) the 
current school year, and (b) their hi^ school career, the perx::entage 
vdio referred to a class ^ch was observed as part of the HOT project 
was 76 (13 out of 17) and 73 (11 out of 15) respectively (see Tables 5E 
and 5F) . 

OJable 5E: Nimfcer of students in Each Sctol Indicating Social StuJies 
as Ifcst Challenging Class Taken in Present SdxxDl Y^r 

Class Sdiool 



HOT Social Studies 
Other Social Studies 
Total Social Studies 

Other Subjects 



* One student indicated two classes were equally challengii^. 
** Oliree students indicated two classes were equally challenging. 



1 


2 


3 


4 


5 


Itotal 


5 


2 


.1 


4 


1 


13 


2 


0 


2 


0 


0 


4 


7 


2 


3 


4 


1 


17 


3 


10 


6 


5 


7 


31 


10* 


12** 


9 


9* 


8 


48 
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'SablB 5F: Ntmi>er of Students in Each School Indicating Social Studies 
as Hc36t Challenging Class Taken in Hi^ School 



Class 

1 

HOT Social Studies 4 
Other Social Studies 1 
Total Social Studies 5 

Other Subjects 4 

9 



School 



2 


3 


4 


5 


Ototal 


0 


2 


4 


1 


11 


1 


2 


0 


0 


4 


1 


4 


4 


1 


15 


8 


4 


•4 


7 


27 


9 


8 


8 


8 


42 



The classes included in the HOT project's saraple were sc icted by 
the social studies department head on the grounds that they engphasized 
higher order thinking (as well as contained a caiposition of stiidents 
according to criteria established by the project staff), C&served 
classes were rated on 13 five-point scales, each of vhich focused on a 
different dimension of thoc^tfulness. Six of these scales were later 
selected as the miniinal criteria for judging the degree of thou^tful 
discourse displayed in a lesson (see i^^pendix 6) • Every lesson received 
a score on these six criteria and mean ratings were coaipited for all 
the lessens observed in each school. Table 5G lists the rank orders 
among the five schools on: (a) their mean score on the minimal 
tiiou^tftOness criteria, (b) the number of students (out of 9) who 
indicated they were required to think very hard in the observed social 
studies class, and (c) the total nuntoer of students' noadnating one of 
the HOT social studies classes as their most challengii^ class, either 
during the present year or throu^out hi^ school. 

Table 5G: Earik Order of Schools on Minimal Bwu^xtfUln^ 
Criteria, Student Deroeptions of Need to Ihirik 
Bard, and Student Rating of Social Studies Class 
as the Most Challenging 



Minimal # students Wost Qialleng. 



School mr Ihink Hard dass 

RANK RANK RANK 

13 11 

2 4 4 4 

3 2 13 

4 13 2 

5 5 5 4 



In all three rankings the top three and the bottcata two schools are 
the same, with schools nuntoer 2 and 5 consistently low on all three. 
And interestingly, the largest difference between schools on the 
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minimal HOT criteria was between the third and the fourth ranked 
schools. The (Spearman's Rank) correlation was 0.6 between the school 
ratings on the minimal HOT criteria aixi the number indicating that 
thinking hard was necessary for conpleting the work, and 0.55 between 
the minimal HOT ratings area the number of studcaits* nominating a HOT 
social studies course as the most challenging class encountered this 
year or in hi^ school. 

Finally, two-thirds of the students indicated (on a five-point 
scale) that their present social studies course (the one identified by 
the HOT project) challenged them to use their mind more than other 
classes they had taken in hi^ school (with one-fifth indicating a lot 
more) . And only five students rated their present social studies class 
as less challenging. (The mean rating was 3.72 with a standard 
deviation of 0.28.) Uiese ratings strongly si?:port the conclxision that 
Hany students find hi^ier order thin3dng cognitively challenging. 

Activitiea and Tasks BaroeivBd as Challencring 

Sli^tly more than three-quarters (77.8%) of the students 
interviewed indicated that they had to think hard to ccnplete the work 
in their present social studies course successfully. Of these students, 
all but three were able to recall and discuss an exanple. 

When students' descriptions of activities in their social studies 
course that were challenging were categorized according to the four 
dimensional coding scheme (see Table 5H), one characteristic within 
each dimension tended to dcaninate. Almost two-thirds mentioned a 
writing task, almost three-quarters describaa tasks or activities 
performed alone, and over half spoke of sn activity or task vhose 
function was to manipulate or make sense of ini'cnnation. With regard to 
the fcurth di m e n sion of content, nearly two-thirds discussed tasks or 
activities in vMch the content involved either ideas or values. 

Because of difficulties in the coding of this data on the content 
dimension (reliability of .62) , student responses to this question were 
re-categorized using an alternative scheme. Hds schema involved the 
four types of content challenges in social studies described by Newmann 
(in press), plus the categories of literal comprehension and recall. 
According to Newmann (in press), major cognitive challenges in social 
studies involve: (a) eitpathy or the incorporation of the experience of 
others into students' own thinking by trying "to see and feel the world 
frcm another's point of view"; (b) abstraction or the use of concepts 
(such as the nature of colonialism and the dynamics of global 
interdependence) or theories (such as the causes of economic 
depression) to try and "make sense of social events"; (c) inference or 
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liable 5H 
Fceguencies and Beroeaitaqes for 
S tudents* Descriptijans of Activities in Social Stuaies Oourse 

that are ChaHencrincy 



type 


no* 


iDarcent* 


writinj task 


21 


63.6 


reading (only) task 


3 


9.1 


discussion 


3 


9.1 


debate, role-play 
questioning 


1 


3.0 


1 


3.0 


unoodeable 


_4 


12.1 




33** 


99.9 


interaction 


no> 


percent* 


self 


24 


72.7 


peers 


5 


15.2 


teacher 


2 


6.1 


unoodeable 


_2 


6.1 




33** 


100.1 


content 


no. 


percent* 


ideas 


13 


33.4 


values 


7 


21.2 


specific topic 


4 


12.1 


facts 


4 


12.1 


uncodeable 


_5 


15.2 




33** 


100.0 


function 


no. 


percent* 


manipulation 


22 


53.7 


C2:^tion 


6 


14.6 


intake 


5 


12.2 


demonstration 


4 


9.8 


mcodeable 


_4 


9.8 




41*** 


100.1 



* Of total nuitiber of responses on each diinension frcan 32 students v*io 

described a challenging activity in their social studies course. 

** One student described two challenging activities 

*** More than one category or function applied to the description of 

seven activities. 
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goinj bsycsnd the given data to develop ejqjlanations, to draw 
concausions, or to make predictions; and (d) evaluation and advocacy or 
develqping and defending '^^ue judgeinents about vAxat is good, ri^t, 
and just in public lifeJ' 

As Table 51 illustrates, the codeable responses were almost evenly 
spread among inference, evaluation and literal cortprehension, with no 
exanples of eirpathy and only a few of abstraction and recall, Uie hi^ 
number of uncodeable responses was due to the lack of discussion by 
many students of the content irvolved in a challei^ing activity, Ohis 
time the reliability, with two coders, was a rcove satisfactory 0,83, 

Inferential challenges described by students included writing 
papers ^mthesising the causes of the Aaverican revolution and 
ej^laining how Ihcanas Jefferson was stereotyped as a defender of 
democracy. Exairples of evaluation were decidiiig vghether or not the 
death penalty is justified and expressii^ and defendii^ one's opinion 
on viiether the motives of the framers of the Constitution were 
patriotic or selfish. Challenges in corprehension cited by two students 
were trying to understand psychological terms used in class and 
readinjs on Athenians vMch were written in their ancient language. 
Examining xdiat malces a dCTOcratic nation and "ocming vqp with a metaphor 
or siraile for a culture" were two of the three exaitples of abstraction. 

Turning to the types of tasks described as chcillenging, over a 
quarter of the 32 students \Aio indicated that they had to think hard to 
ccfftplete their work successfully (and could recall an exaitple) 
described an individual writing task required mam.pulating (i.e., 

organizing, synthesizing, analyzing, etc.) ideas (i.e., the meanir^ of 
the itaterial or principal ideas)- or making value judgements. For 
exanple, a grade 11 student described writing a paper on causes of the 
American revolution: "We had to read several articles and bring the 
material together: economic causes, political causes. We had to state 
an underlyii^ cause and how the other causae related to it." 

In nearly all cases the writing tasks were coaipleted at home or 
outside class, vMch may seem to suggest that homework assignments 
generally are regarded as mDre challenging than activities carried out 
in class. Only two students nominated activities involving direct 
interaction with the teacher. Yet four students described discussions 
or debates with peers as exairples of being challenged (the fifth 
exaitple of peer interaction involved a small qraap writing task) . 
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Table 51 

Desccdpticns of Oanfcent of Activities in Cfoserved Socdal Studies 
Otxtrse that were Reported to be Challenging 

(11=32) 



Type of Challencre 


no. 


percent* 


inference 


7 


21.9 


evalTiation 


7 


21.9 


literal ocsrprehension 


5 


15.6 


abstraction 


3 


9.4 


recall 


1 


3.1 


xmcodeable 


9 


28.1 




32 


100.0 



* Percentages were ccfftputed by dividing the number of stxdents 
respondirig in pach category by the total nuniber of students viio 
reported a challei^ing activity (i.e., 32). 

Note: 1. Tfen students indicate that there v^ere no challenging 
activities in their social studies course. 

2. Biree students said there were challenging activities, but 
they couldn't remearber any exaitples. 

Ihese findings, however, may be biased by the wording of the 
question vMch stated: "Do you ever have to think really hard to 
ccsiplete yoir work successfully in this social staxiies course?". Ihis 
question could have directed students' attention mainly to tasks \^J.ch 
are graded., and in hi^ school social studies^ that invariably suggests 
writing tasks done individually outside class. Inter^tingly, however, 
only four students discussed 'tasks v*iose primary flanction clearly was 
to assess their understanding of knowledge or skills acquired in the 
class. Tasks vMch denanded the creation of an original product, such 
as an original essay or visual metajiior, were more coerairaily mentioned, 
and most students • descriptions did not include any account of 
intentions or efforts to demonstrate knowledge to the teacher. But 
psrhaps, given the wording of the question, this purpose was taken for 
granted and the students focuspd on other aspects of the task. 

In contrast, ^^len students were asked about the types of lessons 
that challenge them to think hard (see Table 4K vMch is discussed 
later) , teacher directed activities were most coramonly identified. This 
tends to si^rt the above suggestion that a finer distinction may need 
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to bs asade in questi-oning students about the cognitive challenge of in- 
class activit/ versus hanework. Unfortunately, no data was collected 
which would enable a ocnparison of the perceived arocunt of cognitive 
challenge of classrocm activities directed by the teacher and academic 
taisks cctpleted outside class, i^roximately half the students in this 
study reported that a hcxnework task in their present social studies 
ocurse had made them think hard and more than b^f indicated that 
certain types of lessons controlled by the teacher also challenged them 
in general to think hard. 

Iteacfcicns to chanemincr Oourse Activities 

All students vflio provided examples of v*ien they had been required 
to think really hard to ccnplete their work successfully in the 
observed social studies class, were asked to describe any frustrations 
and satisfactions that they ejqierienced. Fcustrations wero categorized 
according to the coding scheme for reascais that an activity was 
challenging. While the specific types of ftustrations that were 
recounted varied considerably, well over half the 29 students vAio 
ej^jerienced frustrations gave e3q)lanations that were related to demands 
for higher order cognitive processing (see Table 5J). No other major 
category of e>?)lanations for frustrations was afplicable to more th.^m 
\ "9 students. 
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Table 5J 

Studeants* Iteac±ioais to ChallexTcriiq Activities : 
Fmsfcrations 
(r^=29) 

r io>j ^* frustration frcsm 

cxxtprehen- 2 6.9 difficulty of reacJing material 

sion 1 3.4 student's lack of bactoground in or 

17.2% knowledge of subject 

1 3.4 lade of reading iraterial 

1 3.4 oorrpr^ension but specific reason unclear 

wrk & 3 10.3 quantity of work demanded 

rtmaxY 1 3.4 deinand for concentration owing to 
load uninteresting subject or material 

13.8% 1 3.4 insufficient stuping (for test) 

cognitive 4 13.8 finding correct/relsvant information 

prcx:essing 3 10.3 deciJing cwn position on question or 

demands issue with problematic answer 

58.6% 3 10.3 developing explanations, rea5X)ns and 

justifications for answers or opinions 

2 6.9 creating an original idea 
2 6.9 making inferences 

2 6.9 organizing information 
1 3.4 cogn tive processing but specific demand 
unclear 



teacher 
:o4iaviors 
xO-3% 



peer 

behaviors 
3.4% 



3.4 demanding e5?)ectations and grading 
criteria 

3.4 fear of failing or not performing well 
3'.^ t'Sacher doesn't understand student's 
thinking 

3.4 peers successfully attacking your 
arguments 



uncodeable 



3.4 



* Nurobers add to rore than 29 and percentages add to more than 100 as 
more than cne category of reactions could apply to any response. 
Percentages were canpited by dividing the number of students respording 
in each category by the total number of students ^-Aio r^rted a 
fitistration (i.e., 29). 
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Ttvs3ce was much icore. cxanmonalit/ in the case of satisfactions vAiich 
were categorized using the coding scheme for reasons that an academic 
activity was engaging (see Table 5K) . TV?o-thirds of the 30 students v*io 
reported reoeivirg seme form of satisfaction frcau thinkirg hard were 
intrinsically rewarded by either their perfonnance in cotpleting the 
task or the value they attached to the task itself. Of the fonner 
grcx^), many described their rewards as the feeling of having done a 
good job or of having performod well on the particular task, while the 
rewards for others caine fixm successfully ccjrpletir^ the task. 

OSiese two forms of satisfaction may appear very similar, but the 
first refers to a strong sense of satisfaction with the quality of 
one's work and the effort one has made. For exsraple, one student 
described hew, after spending 10 hours writing a very diffficult book 
review, she read her ccnpleted work and felt "happy with it .. I got 
satisfaction fron a job well done." Whereas the second involves a 
feeling of being pleased with havirg acocwplished the set task 
regardless of the quality or extent of one's effort, as exenplified by 
the student who said: "I succeeded in debating soaaethiiig I didn't 
believe." It is possible (as demonstrated by the responses of two 
students) to e:q?erienoe both these satisfactions from the one task. 
Learning or thinking throu^ sanething successfully on om's cwn was 
mentioned as the intrinsic revv'ard by two students. 

Several students mentioned the satisfaction of developing new 
insists or better understan±ings. As one student eirphasized; 

"I get more satisfaction from truly i:4nderstan±Lng the material and 
having it organized in icy mind than from doing a really good 
report. I get a lot of satisfaction frcen really, really knowirg the 
material and essa.ys are wonderful for helpiiig you ....to really 
understand the material" (due in part, she e>?)lained, to 
researching different sources) . 

Others felt good about either having to think hard or just being 
able to think for themselves. One of these students described the 
satisfaction he rece^ived in English frcm not having to worry about vAiat 
the teacher thinks v*ien he disagreed with his teacher's interpretation 
of the synibolj^ in a story. 
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TaKLe 5K 

Stirients* Reactions to Challerrrim Activities: 
Satisfactioi^B 
(R=30) 

satisfaction f rem 

good grade or ccccDent f ran teacher 
finishing task (sense of relief) 
performing better than others 
passing the test or assignment 





no* 


%* 


instrumental 


5 


16.7 


performance 


2 


6.7 


on task 


1 


3.3 


26.7% 


1 


3„3 



instruniental 
value of task 
6.7% 



intrinsic 


8 


26.7 


performance 


6 


20.0 


on task 




2 


46.7% 








2 


6.7 


intrinsic 


7 


23.3 


valiie of task 






40.0% 


3 


10.0 




2 


6.7 




1 


3.3 




1 


3.3 



3.3 acquiring kncwledge or skills 

that can be xised outside school 
3.3 acquiring infonnation that can be 
used on a future assignment 

performing well or doing a good jcb 
successfully acconplishing task 
6.7 leamii^ or thinkir^ successfully 
on my cwn 

intrinsic performance but specific 
reason unclear 



develc^ing new insists or better 
understandings 
having to think hard 
being able to think for myself 
creating original or unique product 
sharing understandings with others 



uncodeable 2 6.7 

* Numbers add to more than 30 and percentages add to more than 100 as 
more than one category of reactions could apply to any response. 
Percentages were ca!?xited by dividing the number of students responding 
in each category by the total number of students v*to reported a feelii^ 
of satisfaction (i.e., 30). 

Note: 1. Twenty students (66.7%) mentioned intrinsic (i.e,, performance 
or value) satisfactions, and ten students (33.3%) mentioned 
instrumental (i.e., performance or value) satisfactions. 

2. Only six students gave reactions of satisfaction that were 
applicable to r^re than one broad category (e.g., intrinsic performance 
and ir^stn:?ie^intal performance) . 
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Besides these intrinsic rewards, ei^t students admitted rec^ivii^ 
satisfacticai froa extrinsic rerwards that resulted fran their 
performance cxi a task. For five of these students the scfurce of reward 
was a good grade or words of praise fron the teacher. And soroewhat 
surprisingly, no student even in the axtpetitive upper track classes 
- spoke of any sort of recognition fron peers as a source of 
satisfacticai, and only one inentioned being rewarded by doii^g better 
than his classmates. Satisfactions derived from the instnmantal value 
of a task were not ocatnmon. 

Althcu^ the students were invited to talk about as many 
fnastratione as they could remeniber, the vast majority mentioned only 
onB. In contrast, most students described more than one type of 
satisfaction that they derived fron thiricing really hard. Among those 
vdio offered more than one e^qplanation or description of frustration, 
there were caily two students whose responses applied to two different 
major categories. OMs time a similar trend was evident for 
satisfactions vdiere the responses of only six students fell into more 
than one of the four major categories. In other words, for 80 perx::ent 
all their satisfactions were related to ots of the followir^: 
instrumental performance, intriiisic performance, instrumanfcsil value or 
intrinsic value of the task. But more inportantly, only 20 perx^ent of 
students derived satisfaction only for instrumental reasons. 

ChallencrLncr Qusstions or Tasks in Observed Social Studies Cl asses 

In addition to general questions about school subjects and the 
social studies course included in the HOT stud/, students were also 
asked on each of the three visits to reflect on a particular social 
studies lessai observed by a researcher, ^he responses to the question: 
"What questions or tasks in that lesson, if any, made you really think 
or use ycur mind?" did not fit the categories used for analyzirg 
students' more general descriptions of challenging activities in social 
studies. So, instead, this data vias categorized solely by relying on 
the aJ.temative scheme enployed for coding the content ccanponent of 
challenging social studies activities, nairaly the extension of 
Newmann's (in press) four types of diallenges. 

As Table 5L indicates, over 80 percent of the responses could be 
categorized within one of the four content challenges that demand 
higher order thinking (i.e., inference, evaluation, abstraction, and 
empathy). Ohe most frequently mentioned exaiiples were of inferential 
challenges where stu d ents had to go beyord the data to draw 
conclusions. For exanple, one student described trying to figure out 
vdiat Er^land or France really did, if anything, to develop their 
colonies, while another student was challei^ed by the self -initiated 
task of explaining why the standards and quality of food we enjoy today 
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are so different frcm those during the t±m of the loeat packers' 
cxsntroversy. 

Table 5L 

Descriptions of Oaestions or Tasks in cabserved Socdal Studies 
Lessons that wsre Reported to be CteJJLenqing 

(n=99) 



Tvre of Challerxie 


no. 


percent* 


inference 


34 


34.3 


evaluation 


26 


26.3 


abstraction 


21 


21.2 


recall 


6 


6.1 


empathy 


4 


4.0 


literal ccarprehension 


2 


2.0 


vmcocJeable 


_6 


6.1 




99 


100.0 



* Percentages were ccarputed dividing the nuitiber of student 
responses in each category by the total nmJber of responses (i.e., 99) • 

Note: 1. In 22 cases students indicated that there was no chal'-'-^ tfir^ 
question or task in an observed lesson. 

2. Twelve students were not taking the saina social stutdes course 
(i.e., the one included in the HOT saitple) on the third visit and hence 
were questioned on only two observed lessons. 

3. Tliree students were absent frctn the observed lessons on one 
visit and hence were questioned on only two lessons. 

4. More than one i^pe of challenge applied to one response. 



CJonsistent with the findings already discussed on the reasons 
students find social studies classes to be challenging, a number of 
responses referred to questions or tasks which required naking a 
valuejudgement. Ejcairples cited by students included decidij^ vdiether 
civil disobedience is justified, vAiether Jefferson acted properly in 
establishing an embargo, and vAaethe*. nullification could ever be 
justified. 

When identifying challenges in specific lessons, students 
described problems of abstraction (in 2nakli>g sense of a conc^ or 
theory) more frequently (one in five) than when discussinr challenges 
of the social studies course (one in ten) or of school subjects 
generally (no inention of conceptual or abstraction prdblems). 
Distinguishing between ccaounism and scxjialism, understar.diog the 
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checJcs and balances of presidential and cxangressional power, and 
defining "ethnic plxaralism" are some of the challenges in abstraction 
reported by students in these lessons. A possible e5$)lanation of vdiy no 
students inentioned abstract concepts as a reason for finding social 
studies a challenging subject is that these challenges may be irregular 
and short-temi/ rather than a continuing cognitive challenge as mic^t 
be the case for developing and defending ejqjlanations or opinions* 

Only a few students spoke of having to think hard to gain an 
eirpathLtic feel for e)5)eriences occurring in a different time ana/or 
place. Specific exairples offered were: trying to vmderstard vdiy 
security guards in a prison have certain attitudes toward inmates, 
'hmdergtanding the concept of an historian's views being influenced by 
the tiine in vUcdcti they wrote", and trying to see "the way the elite see 
and the way the f aru^ sees things" (at the Constitutional Convention) • 
Ihe low number of etpathy challenges could be attributable to teachers 
not presenting task? aimed at this objective in a challenging form, but 
instead using films, stories or lectures for this purpos3 v*iich may not 
require students to actively develop answers to challenging questions. 

In addition to the above hi^er order thinking challer^es, only a 
few responses indicated that lower order cognitive tasks were 
challenging, ^he majority of these involved the recall of previously 
encountered infonnation, with the remainder being concerned with 
carprehending the literal meaning of new infonnation. 

Only two students found nothing diallengii^ in any of the three 
observed lessons on vMch they were questioned, three students reported 
nothing challenging in two of the threa lessons, and another 10 said 
there was do question or task that made them think hard in one of the 
observed lessons. And interestingly, no student described the sam type 
of challenge in all three lessons. 

Re^±ion3 to Challencrina Questions or Tasks in (^served Lss sms 

As in the case of challenging course activities, students \Aio 
indicated that a question or task from a lesson observed fcy a 
researcher made tiiem think haird, were pg V g^c i to describe ariy 
frustrations and satisfactions th^ encountered in responding to that 
particular question or task. It is interesting to conpare the responses 
to this question with those to the equivalent question on reactions to 
challenging course activities. Such a conparison provides a check on 
the consistency of student responses and an indication of the validity. 

Ihe percentage of responses in each of the major categories of 
frustrations from dealing with diallenging qusstions oir tasks (sss 
Table 5M) was very similar to those for challei^ing course activities. 
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cxarprehen- 
sion 
17,0% 



work & 
inemory 
load 
6.4% 

ccgnitive 
processing 
demands 
48.9% 



teacher 
behaviors 
6.4% 



peer 

behaviors 
10.6% 

vmcxxieable 



Table 5M 

Ra^^icns to Qiallarmiry Questicans or Tasks 
in Cfcserved Social Studies Lessons; Ecustraticgis 

(n=47) 

rK).* %* frustration f rcan 

3 6.4 lade of sufficient information 
2 4.3 difficiuLty of reading material 
2 4.3 student's lack of bacihgrouixi in or 
knowledge of subject 

1 2.1 ccKfprehending literal ineaning of cqtk:^^ 

2 4.3 demand for accessing previous course 

material 

1 2.1 demand for memorizing present material 



8 17.0 deciding am petition on question or 

issue xdth problematic answers 
5 10.6 having to make abstracticais (to make sense 
of information) 
3 6.4 developing e55)lanations, reasons ani 

justifications for answers or cpinicms 
3 6.4 making inferences 
2 4.3 finding correct/relevant information 

2 4.3 taking the perspective of aix>ther person 

1 2.1 lack of clear guidance in imderstanding 
material 

1 2.1 tmcertainty about teacher's response 
1 2.1 lack of opportunity to create own 
solutions 

1 2.1 insufficient tim for discussion 

3 6.4 trying to convince peers of your argument 
1 2.1 peers quicker at answering questicais 

1 2.1 disagreements in peer group 

5 10.6 



* Numbers add to more than 47 and percentages to more than 100 as more 
than one category of reactions could apply to ai^ response. Etercentages 
were carpited by dividing the irurriber of student responses in each 
category by the total number of responses (i.e. , 47) . 
of the interview schedules. 
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Ten percent fewer students reported frustrations related to hi^ier 
order cx)gnitive processing demands in cibserved lessons, althou)^ this 
category still represented approximately 50 percent of the students. 
Within this category, more students indicated fnostrations in observed 
lessons (than in the \diole course) with deciding their position on a 
problematic issue. Ihis may be the result of dealing with new and 
COTiflLicting information in a lesson, \Aiereas assignments to a large 
extent involve writing about an issue that has already been discussed 
and that students have had seme tiine to think about. As one student 
e5q>lained in discussing a class argument on the issue of civil 
disobedience, "it was a little frustrating - cause there's no real 
answer. Part of it may be b^aiise since it's the first time we've 
discussed it, it's a little confusing." 

On the other hand, fewer students erKx>untered frustrations with 
finding correct or relevant information in the obser/ed lessons. Ttds 
is not surprising as students' exaitples of having to think hard to 
ccaooplete work successfully in the social studies course generally 
involved writing papers, vftiich, of course, require searching for 
appropriate information. 

The number of students v*io described satisfactions derived frcati 
their intrinsic performance was noticeably less - approximately one-- 
third instead of half - on questions or tasks in the observed lessons 
than on exairples of assignments done throfu^out the course (see Table 
5N) . But we would expec± nore satisfaction v^ien the students can choose 
an example of the most challenging activity or assigiroent encountered 
over the span of the entire course. Iheir performance on such a major 
and demanding activity is more likely to create intrinsic satisfaction 
than on jiist a question or task from a lesson selected at random. 
Nevertheless, the two most common descriptions of satisfaction were the 
same in both cases, namely: successfully acconoplishing the task, and 
performing well or doing a good job. 

Ihis last description, however, did drop from being mentioned by 
27 percent of the students in relation to course activities to oiily 
ei^t percent in relation to observed lessons. A student \iho felt he 
did a good job on a question in an observed lesson e3q)lained that 

"it was satis:^ing to knew vdiat I finally believe. When I decide 
based on considering evidence and different points of view, you 
know you've made a good decision becaxase you've seen both views 
.... The reward is v4ien, after you reach a concliasion, you can 
articulate your position, knowing you've examined all sides of the 
issue." 



153 



V - 23 



rable ST 

Rsacfcicns to CSiallencpm Caestians or OSasks 
in Cjasetrved Social Studies Lessons; SatisfafAi cig 

(n?=63) 



satisfaction from 





IK)* 




instrumental 


7 


11.1 


performance 


6 


9.5 


on task 


2 


3.2 


27.0% 


1 


1.6 




1 


1.6 


instnmental 


1 


1.6 


value of task 






1.6% 






intrinsic 


13 


20.6 


performance 


5 


7.9 


on task 


3 


4.8 


31.8% 








2 


3.2 


intrinsic 


15 


23.8 


value of task 






34.9% 


4 


6.3 




2 


3.2 




2 


3.2 




1 


1.6 


uncodeable 


7 


11.1 



good grade or ccjnment fron teacher 
performiiig better than others 
receiving recognition frcsn peer's 
teacher failing to prove roe wrong 
seeing that peers think the same as 
me 

acquiring knowledge or skills that 
will be useful in the future 



successfully accoatplishing task 
performing well or doing a good job 
learning or thinking successfully 

on iry own 
being able to convince others of ny 

argument or defend my opinion 

developing new insists or better 
understandings 
having to think hard 
sharing understandings with others 
hearing appealing ideas 
being able to build further ideas 



11.1% 

* Numbers add to more than 63 and percentages to more than 100 as nore 
than caie category could apply to any response. Percentages were 
ccnpited by dividing the number of student responses in each category 
by the total number of responses (i.e. , 63) . 

Note; 1. Fourty two student responses (66.7%) mentioned intrinsic 
(i.e.^ perfonnance or value) satisfactions^ and 18 student responses 
(28.6%) mentioned instrumental (i.e.^ performance or value) 
satisfactions. 

2. Only four students gave ireactions of satisfaction tihat were 
applicable to more than one broad category (i.e.^ intrinsic performance 
and instrumental performance) . 
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Such an escamination/ hcie^er, is generally rx>re likely to cxxur 
throu^ the process of ccarpleting a inajor assignment than dealing witha 
question or task in one relatively brief lesson. Hixjis, the predominance 
of assignments as the exaitples of challenging course activities is 
sufficient to e35)lain the greater occurrence of satisfaction fram 
perfoirming well in this context coirpared to that derived from 
challenc/ing questions or tasks in a lesson. 

On the other hand, developing new insists or better 
understandings, such as "it gave ine more knowledge of the different 
categories in v*iich the psychologists' approaches could be placed", 
remained a scwrce of satisfaction for alnost one quarter of the student 
rgjonses. Even if students only "understand it (how early man survived) 
a little better" after an in-class task, they at least feel they "can 
say you learned scaiBthing today." And vMle irost students' insists 
related to a new or increased understanding of content, sane 
ejqperienced inore far reaching insists about people and thcjir views: 

"It shewed itB that people have diffeirent views and that people 
think differently. . . So now I try to have a icore open view ba':ax2se 
different people interpret in different ways. So majiDe it will inake 
ine a more open person to \diat people think." 

In addition to intrinsic value cf the irask, there was also a 
noticeable siinilarity in the percentages of students on the otlier two 
inajor categories of satisfaction (i.e., instrumental perfomonce and 
iiistrumental value) between the reactions to challenging course 
activities and the 2:eactions to challenging lesson tasks. Ihis 
consistency confirms that approximately one third to one half of the 
students in this stuffy derive satisfaction from their intrinsic 
perfonnance, inore than one third from the intrinsic value of the task, 
and only approximately a quarter from their instrumental perfonnance on 
a challenging task. 

Types of Lesson Formats that ChallervaB Students to giink B^rd 

Students were requested to indicate the general types of lesson 
formats (in all subjects) that challenged them to think hard (see Table 
50) ^ Two^ teacher directed activities, teacher-led class discussion and 
qoestionii^ class on understanding of subject matter, had the hi^est 
frequency of checks. In both cases, students ejqolained the need to 
ooiKi^ntrate on imaerstanding the discourse so that they would be 
prepared to answer any questions if called vpon. As one student 
e3?)lained, "you have to be thinking all the tiite vto you have a 
teacher \gho doesn't just call on volunteers." Furthermore, tm other 
students eirphasized that "teachers yAio make you think the most keep 
picking on you if you don't give a reasonable answer." 
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Table 50: OJ^pes of Lesscsns Students Report as Challerfing Tbnssi to 

TSmdc Baxd 
(n==43) 



Lesson Format Frequency Percentage 



1 


class discussion led by teacher 


21 


48.8 


1 


teadier questioning class on 








imderstanding of subject matter 


21 


48.8 


3 


working on a task by myself 


19 


44.2 


4 


class discussion led by students 


12 


27.9 


4 


d^ends on the class or teacher 


12 


27.9 


6 


lecture 


9 


20.9 


7 


working on a task in a small graap 


8 


18.6 


8 


working on a task with a friend 


6 


14.0 


9 


film 


3 


7.0 


10 


other 


2 


4.7 


11 


nor 


1 


2.3 



l^parently, teacher questioning creates pressure by putting 
students "on the spot", "vdiereas in class discussions led by students 
you don't have to say anything" or "you can just say \diat you think." 
Consequently, althou^ ranked fourth, only just over one quarter of 
respondents find class discussions led by students chaillenging in 
contrast to the two-thirds viio indicated that they are interesting and 
worthsMle. The challenge for these students is visually in role plays 
or debates vtoe they have to develop arguments "for something you 
don't really believe in" or "to convince others about your opinion." 

Closely behind teacher-led questioning ard discussion was working 
an a. task by yourself when, according to a number of students, all the 
thinking had to be done on your own. Not only do you have to get along 
"without the benefit of others' ideas and views," but also "there's no 
feedback vtien you work alone." On the other hand, working on a task in 
a small groi^) or v/ith a friend is far more engaging (rated so by more 
than half) than challenging (as indicated fcy less than one in five). 
Several _ students admitted that groi^rk was fun, but often involved 
discussing non-task matters. But the major reason advanced for graap 
tasks being less challengir^ was the opportunity they provided to rely 
on other people to do the thinking. Yet some students found it 
challenging in a small graixp task to connect xxp different people's 
ideas, vMle others suggested that overcoming the inevitable 
disagreements and arguments (over procedures as vrell as substance) was 
always a challenge. 
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Lectures were rated engaging and challenging "77 rou^y equal 
percentages, but only three students thou^t a film made you think hard 
ocsrpared to 18 students vdio rated it interesting and worth^Mle. 
lectxare^j are hard for seme students because "some things go by you, but 
the teacher can't step each time something is covered that you don't 
understand." Said another way by another student, there is not the 
opportunity for things to be repeated. 

Many students indicated that the type of lesson which roaJtes yoa 
■chink hard depends on the subject or teacher. Again, one interesting 
distinction was between subjects vfliere there are ri^t answers and 
those involving opinions or value judgements: 

"If one thing is ri^t - as in math and science - you have to 
think harder. That is, trying to rationalize P=MA is harder 
than figuring out my opinion on the progressives." 

Types of Writing T»iskB Rxmd Challerxriirr 

As well as the different t^,pes of lessen or instructional formats, 
students' reactions also were sou^t to the challenge of different 
types and different objectives of writing tasks. Almost 80 percent 
indicated that writing a paper challenged them to think hard (see Table 
5P). Percentages then declined as the type of writing decreased in 
length or substance. Thus, writing paragraphs or a short essay was 
challengir:g for about half the students, making an outline the same for 
one in five, and writing one or two sentences made less than oni3 in 10 
think hard. 

Table 5P: Types of Writing Students Report as Challenging Ihan to 

Oliihk Bard 
(n=43) 



Generally students report that the major challeyyge of writing 
papers is finding enou^ information, especially on tqpics that they 



Rank 



Type of Writing 



Frequency itercentaqe 



1 
2 
3 
4 
5 
6 
7 
8 



writing a paper 34 

writing paragraphs or a short essay 22 

depends on the topic or subject 14 

making an outline 9 

taking notes (frcm a lecture or book) 7 

writing one or two sentence answers 4 

other 3 

none 2 



79.1 
51.2 
32.6 
20.9 
16.3 
9.3 
7.0 
4.7 



15: 
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feel they Joicw little about. And not only is sufficient information 
required, but "you have to think of different ideas that will bind 
together ... and that will si:?)port your thesis^." l^nother difficulty 
iftentioned by several students is organizing the paper, for exairple 
"into a topic sentence, logical order of paragraphs, introductory and 
concluding sections." 

Even a coi^le of paragraphs is tou^ for san^e students, one of 
v^cDi candidly stated: "You get tired of writing so you jxost put down 
anything. A paragraph isn't too hard, but anything longer is hard to 
find enou^ things to write down." Yet for other students shorter 
writing tasks are harder, because of the need to be succint and to "fit 
all your ideas into a short space." One student, vdio has "lots to say", 
finds it hard to use only specific facts, to oinit uiJitportant details, 
and to not be too genersil. 

Making an outline provides the challenge apparently of deciding 
to include and the order in vMch to include it, vMle taking 

notes frora a lecture (but not a book) demands thinking "because the 

teacher won't stop and you have to pick out itrportant points quickly." 

And, of course, you have to be "able to read your own writing 

afterwards*" 

M?ny students indicated that air^ of the listed writing tasks can 
be challenging depending on the subject or particular assignment. For 
one student, "the organization of other people's ideas that you leam 
is easier than expressing your cwn thou^ts." Somevtot in contrast was 
the student vfao found papers in social studies hard because the topic 
is always one that is just being learned, vfliereas in English often the 
information is already known (e.g., your experiences) . 

In re^nse to the question concerning the nature or objective of 
writii^ assignments that challenge you to think hard, five of the six 
listed were diedced by a third or more of the students (see Table 5Q) . 
Writing vMch involved taking a position on an issue and defending it, 
e>q)laining a concept/principle/ theory/problem or issue, and reporting 
on independent research were each rated as hard by approximately half 
the students. The first was generally said to be challenging because 
you have to think of reasons and "you may like scawething but not know 
hew to defend it." Or even if you do have reasons, "you have to be sure 
of vdiat you are defending or opposing and that causes you to prepare 
the utmost to cover all the bases Sf) you will not be caui^t off guard. " 
E>5)lanations of concepts, principles, etc. v^re reported to require 
thinking in order to put them in your own words, especially vtoen the 
"sources are in conplex terms you have to siitplify." Ihe most 
descriptive explanation of the cognitive demaniL of report writing 
involved the need to include both sides, or the pros and cons, of the 
positions on the issue beirg researched: this meant "you have to 
continually think \^^lether you have covered everything." 
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Table 5Q: Subject of Writing AssigrEsnts StuSents Kepart as 

CbaHenjirg Tben to ndbnkBard 
(n=43) 



Rank 



T ype of Writing Assignment 



Frequency Percentage 



8 

9 



5 
6 
7 



3 
4 



1 



taJdng a position on an issue and 
def encJing it 

es^jlainlng a conc^, principle, 
theory, problem or issue 
r^rting on independent research 
creative writing (making iqp stories 
or poems) 

describing iny es^jeriences or feelings 

depends on the tcpic or subject 

summarizing matericii covered in class 

or readings 

other 

none 



21 
20 



18 
14 
10 



& 
2 
1 



41.9 
32.6 
23.3 



18.6 
4.7 
2.3 



48.9 
46.5 



53.5 



Creative writing is hard for many students owing to either the 
demand to be original or the lack of concrete material with which to 
work: "you start frcaa scratch and make it iqp by xxsing your 
imagination," whereas in other forms of writing "you have something to 
start with." Ej$)eriential (i.e., describing ejqjeriences or feelings) 
writing, as described by one student, demands getting "down to your 
feelings, lodkir^ at hew you really feel, it's not that easy." But the 
concern of another student was revealing too much of his feelings 
"because it gets into the wrong hands." Unfortunately, for this student 
making decisions about '^l4lat to tell" is the .hard part. 

By contrast, one student never found creative writing hard "as it 
comes from within yoii. You're edso not concerned about vdiether you have 
got things ri^t or wrong as you knew you're not going to be graded on 
facts or vdiat you have learned." The eirpliasis on giving (ri^t) reasoris 
in students' ejqplanations of the challenge of eiipository writing 
suggests that others may not find creative writing challenging for this 
same latter reason. 

Trying to identi:^ the most important facts seemed to te the ma^in 
challenge for the few students vftio indicated that summarizing material 
made 'them think hard. Typifying many of the reasons offered for v*iy the 
cJiallenge of writing assignments depencJs on the topic or subject was 
the particularly optimistic reponse that: 

"Anything that's worthvMle challenges you to think. Creative 
writing allows you to choose from unlimited eptions. Writing a 
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story involves thinking of an original plot, ani that's hard. 
Taking a position challenges you to articulate your thou^ts 
that are often ftizzy to begin with. You have to understand to 
e^qplain a concept, and this forces you to think." 

SCMHRRg ; 

Over a third of the students interviewed in this stucty ncffoinated 
social studies as the itiost intellectually challenging subject they had 
talcen in hi^ school, \Mle a few roore said that science or mathematics 
was the most challenging, and one in six noaninated English. More than 
half the students who selected social studies gave the reason as either 
having to address ill-defined problems or questions with prxSDlematic 
answers, or being required to make inferences.-^ The other half were 
equally divided in their reasons between liaving to overcome 
cccprdiension difficulties and coping with the work or memory load, 
especially the quantity of work demanded. By contrast, half the 
students challenged the most by science/math attributed the reason to 
solving well-defined problems with single, correct answers. Most of the 
others found the conc€^ in science or math difficult to understand. 
The quantity of work was the contributing reason provided by half the 
students reportii^ Er^lish to be the most challenging subject. 

Two findings strongly suggest that many students find hi^er order 
thinking challengii - First, a large number of students indicated that 
the course iiKiluded in cur stuc^ of social studies departments 
emphasizing hi^ier order thinking was their most challenging class. 
Second, there was a reasonably hi^ correlation between school rankings 
on the number of such courses nominated as the most challenging and the 
srankings on the extent of hi^ier order thinking observed in dassrxxms. 

Three in four students indicated that they had to think hard to do 
their work successfully in social studies. The majority mentioned 
individual writing tasks costpleted outside class as exanples of a 
challenging activity. The type of content challenge selected for social 
studies courses in general tended to involve inference, evaluation cr 
literal ccirprdaension. Inferential and evaluation challenges also were 
the most ccsnmon types described by students in relation to questions 
and tasks that made them think hard in a lesson observed by a 
researcher, and problems of abstraction were also quite often noted in 
particular lessons. 

Students Tvere divided in their ratj, of the degree of difficulty 
encounteroi vihBn their social studies teacher posed a problem or 
question v^tiere there was no single correct answer. Approximately half 
rated such problems quite difficult, while the other half ratal them 
not difficult. This is consistent with the findings on reasons for 
different subjects being challengii^ vAiere seme students selected 
social studies because of its ill-defined prxiblems with problematic 
answers, vMle others selected another subject (usually science or 
mathematics) because of its well-defined problems whicih they found more 
challenging. 
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Both teacher-led class discussions and teachers questioning 
members of the class on their understanding of the material r^rtedly 
made almost half the students think hard because of the need to be 
pr^ared to answer a question ^-ihen called t^xai. Almost as many students 
indicated that workir^ in class on a task by oneself was challenging: 
for scane because of the lack of irpit or feedback frcro otheris. 

The vast majority of students reported that writing a paper was 
diallenging, usually because of the amount of information that had to 
be found. In genered, the shorter the type of writing assignment the 
fewer the number of st.-udents vAio found it made them think hard. As for 
different types of writing assignments, taiking a position on an issue 
and defending it, explaining a conceptual-theoretical material, and 
reporting on indepenclent research, were each checloed as making them 
think hard by approxiriately one in two students. 

The Conclusion to Oiapter VI will relate findings on student 
perc^jtions of cogm.tively challenging work to their perceptions of 
engaging academic wo3:k. 



161 



VE 



STOEENT HERSEBCEIVES CN ESKJ^GEING CnRRICDnM 
liotsert B« Stevesiscn 

jNssDoacncK : 

A comnan theme in much of the canteitporary literature on secondary 
schools is_ that students generally are indifferent to schoolwork and 
exert a minimal amount of effort (Bqyer, 1983; Cusick, 1983, 1973; 
Everiiart, 1983; Goodlad, 1984; Powell, Farrar, & Cchen, 1985; Sedlak et 
al, 1986; Sizer, 1984). Teachers strongly agree, rating student 
passitivity and lack of interest as their worst prtSjlem (Hanpel, 1986; 
Tye, 1985). This academic apathy or low-level participation in 
schoolwork, however, is a syirgotom of a deeper condition of student 
disengagement (Natriello, 1984). 

_ As described in the recent literature, the prcblem of 
disengagement in rigorous academic work afflicts students of all levels 
of academic ability and from all socioeconomic and racial classes 
(Goodlad, 1984; Sizer, 1984; Cusick, 1983; McNeil, 1983). Although the 
hi^ier drop-out rates for Hispanics, blacks, and youth fron households 
of lower socioeconcffldc status (U.S. General Accounting Office, 1986) 
suggest that students fron such backgrounds are more likely to have 
strong negative attitudes towards school and to be alienated fraa the 
institution, lack of academic engagement is certainly not confined to 
alienated students in general or these students in particular. In fact, 
according to a recent report there is a long tradition of the vast 
majority of hi^ school students being indifferent +-o academic 
learning, with the problem claimed to have beccne exacerbated in the 
last twenty years (Sedlak et al, 1986). Ihe lack of ei^gement among 
most students is manifested in a more neutral attitude where schoolwork 
arouses neither negative nor positive feelings and a passable but not 
utmost effort. 

Althou^ disengagement has been described extensively in the 
literature on secondary schools, vtiere the problem seems to be itost 
serious, the authors of a recent review concluded that there apoears to 
ba no study that has examined the other end of the continuum:' student 
engagement in academic work (Mosher & MacGowan, 1985) . ^et, in order to 
tackle the problem of disengagement, it is critical to landerstand vdiat 
engages hi^ school students in academic work. More precisely, we need 
to identify the conditions that will facilitate a posi+:ive attitude and 
the exertion of a committed or maximum, rather than apathetic or 
minimal, effort. 
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Given the lade of research on this subject an appropriate starting 
point would seem to be the perceptions of students regarding what kirrl 
distinguishing <±arac±eristics of sane hi^ school classroons (as the 
settings for academic work) vMch students associate with er^gement? 

A number of studies have suggested that schools (i,e, classrxxras) 
vary ccaisiderably in the extent of student er^gement (Rutter et al, 
1979; Vfehlage et al, 1980; Wjame, 1980), lEherefore, this study sou^t 
to identify the characteristics (e,g, curriculum content, format, 
instructional org^zation), as perceived by students, of lessons and 
academic tasks wiich generally evoke engagement in thinking and 
learning. For this purpose student er^agen^ent in academic leamit^ is 
defined as a learning situation (i.e. a class lesson or academic task) 
in v*iich: a) the student makes a serious or canmitted effort t:o master 
the knowledge and/or skills intended to be developed, and b) -the 
student values the work itself or the actual process ana/or substantive 
outocroes of learning (rather than the institirtlonal rewards vMch mi^t 
accrue) as meaningful and worthwhile, or finds the topic intrinsically 
interesting. 

This chapter presents the findings in relation to the two r^earxih 
questions: 

(a) What kinds of academic work do hi^ school students report as 
being engaging? and 

(b) What reasons do th^ give for finding such work engaging? 

Students were questioned about "engagir^" academic work In several 
different leamir^ contexts. Ihese contexts consisted of any subject or 
course the student ncsnlnated as particularly erQaging, the social 
studies course that was observed by a HOT project researcher (both in 
general and within specific observed lessons) , and any class vAxere the 
student had e55)erienced in-depth work. Specifically, students were 
asked to describe: (a) the most engagirg course or subject they had 
taken in the current school year, (b) exanples of the most engaging 
activities or tasks from a particularly engaging class (in any 
subject), (c) examples of engaging lessons or assignments in their 
present social studies course, (d) specific questions or tasks from 
three social studies lessons observed by a researcher vMch they found 
engaging, (e) anythiixf, if at all, they found er^ging about an^- in- 
depth work they had e5^)erienccd in school, (f) teacher beliaviors 
associated with ej^ging elcis^(es), and (g) types of lesson formats 
they generally found engaging. 

In all cases reasons were sou^t from students as to why the 
exaitples they provided were engaging. It was from these reasons that we 
hoped to^ learn the most about students ^ perspectives on the 
characteristics of engaging academic work* 
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Most Interestincr sand Tfarthwhile Oaurse 

When asked \*at was the nost interesting and worthwhile class they 
had taken durir^ the present sdiool year, almost half of the students 
indicated a social studies course (see Tible 6A). Ihe social studies 
courses described as the itost engagi >g ccnprised 10 history^ three 
general social studies, two econonics, two psychology, and one law. 
English, including two speech classes and a writiiyj class was the only 
other subject area naninated by irore then ten percent. 

2he students interviewed, hcwever, carae frcv^ five schools that 
were selected for the attention their social standi es di.^^)artanents gave 
to fostering hi^ier order thinking (HOT) . rvrthennore, in four of these 
schools the prli>cipal considered that d^>arboent to have the best 
teachers. So, vhile these schools are not representative of high 
schools In general and therefore the social stixiies esqjeriences of 
their students is not representative, these findings do iixilcate that 
mai^ students fli^d social studies particularly ei^girg. IhiB findii^ 
is particularly encouraging given that other studies have found that 
social studies is rated lew in interest level by secondary students: 
for exaitple, in one study only 13 percent indicated that social stu^'-*^ 
was their favorite subject (Shau^inessy & Halad/na, 1985) . 

Students were as k ed to e>5)lain the reasons vAiy they found the 
cojnrss they ncminated engaging. Althou^ engagement has beer, defined in 
this study in intrinsic terms (i.e., the subject matter, or; the process 
or cutcanes of learning are valued for their own sake or regarded as 
intrinsically interestirg) , students were asked to indicate a course 
that was interesting and worthvMle. in other words, as students 

Table 6h: Subject of Kbst Ergaging Class Taken in Present 

School year 
(nF43) 

students 



Rank 


Subiect 




Percent 


1 


Social Studies 


18 


41.9 


2 


English (incl. Speech) 


8 


18.6 


3 


Science 


4 


9.3 


4 


Matheitatics 


3 


7.0 


4 


Foreign language 


3 


7.0 


6 


IntJustrial Arts 


2 


4.7 


6 


Hi^ TedhivJlcgy Lab 


2 


4.7 


8 


Art 


1 


2.3 


8 


Health 


1 


2.3 


8 


Ehotography 


1 


2.3 






43 


100.1 
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e55>lairied the reasons for their engagement they could characterize 
classes as "inten^sting" for either extrinsic or intrinsic reasons^ or 
as •'worthwhile" for instrumental or non-instrumental reasons. 

As T&ble 6B illustrates^ over three-quarters of the students gave 
at least one reason that referred to the intrinsic value of the class 
(i.e./ they were engaged by either the content^ processes or outcomes 
of learning in that class), ihe most comrnon single reason in this 
category (mentioned by nearly half the students) was that the subject 
matter was intrinsically interesting, often owii^ to its perceivoi 
relevance to the real world or, more specifically in the case of social 
studies, to current issues and events. Chemistry, for exarple, was 
described as engaging because "it answers questions you ccme across in 
day to day life: for example, ifdiy is frozen ice less dense than water?" 
One student found U.S. History engaging because "the material relates 
to our current political and econaidc system." Yet for another 
student, history was intrinsically interesting, not for its 
contenporary relevance, but because "we deal with very informative 
tcpics and real characters vftiom ycu can relate to - you can relate to 
v*iat happened, hew people felt and \diat th^ went thrro^, for exanple, 
in colonial tiines." The particular tcpics that were mentioned as 
intrinsically interesting m any subject tended to be idiosyncratic 
with no tcpic engaging several students,, 

Bie other ocaranon reasons in this catugory of intrinsic value 
involved the c^jportuniiy to actively participate in the class, by 
discussing one's own op.lnions or ideas, by creating an original 
product, by manipulating information or materials (e.g., technical 
drawing ai>d scientific ins1:ruments) , or by working on a challenging 
task. Ihese four categories combined represented a reason given by more 
than two out of every five st»3dents questioned. Ihe engagement power of 
being able to esqpress qpinions or ideas in class is conveyed by the 
follcwing three quotatidriis from students: 

"Every week we had maybe two speeches to write ai^ 
everything. . . .they were tcpics that everybody could relate 

to like favorite teachers and it gave us the opportunity to say 

v*at we have to say about teachers, about vAiat we didn't like about 
school ...things you normally don't have the opportunity to say.-' 
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Table es 

Reascgs Subrlecb or course was ifhs Mast Enoarim 



instrumental 
perf orroance in 
class 2.3% 
instrumental 
valiie of class 

14.0% 
intrinsic 
performance 
class 7.0% 
intrinsic 
value of class 

79.1% 



teacher 
behaviors 
41.9% 



no.* %* 
1 2.3 



5 

4 

4 

2 
2 



1 
9 

5 

5 

2 
1 
1 

5 



(n=43) 

subject was easy to learn 



6 14.0 relevant to future goals 
3 7.0 performed well 



19 44.2 intrinsically interestii^ subject 
matter 

10 23.3 opportunity for discussion or 

contriixition of cwn ideas/cpinions 
11.6 leanied nsv thii^gs or gained new 

insists 
9.3 opportunity to laanipulate 
information or materials 
9.3 opportunity to create xmique 
product 

4.7 cognitively cihallengirQ task 
4.7 ijpportunity to hear ideals/opinions 
of others 

2.3 opportunity to share understaniLngs 

with others 
2.3 logical subiect mtter, ma':3S sense 
20.9 instructional practicas ^made 

subject interesting, ciiallenging) 
11.6 attitude towards students (caring/ 

gave responsibility) 
11.6 personal characteristics (likeable, 

sense of humor, enthusiastic) 
4.7 demanding academic es^jectations 
2.3 knc^7la2geable about subject 
2.3 provided variety (of activities) 
11.6 specific teacher behavior vinclear 



2.3 serious attitude to learning 



peer i 
behaviors 2.3% 

* Numbers add to more than 43 and percentages add to more than 100 
as more than^ one reason could apply to any response. Percentages 
ccaiputed by dividing number of studaits responding in each category by 
total naniber of students (i.e., 43). 
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"Bie teadier lets us do more, he gives us itore freedcaa so that it 
feels more like hi^ school should be, \Aiereas other classes seem 
more like junior hig^i. He starts us off ard then the class teaches 
the class, but he corrects us v4ien we are wrong or need help so 
that we are not left alone." 

"Everyone can say vfliat they want to say. In large groups you can 
say vAiat's on your mind. You can say at the end of the class that 
you contributed to the answer." 

Similar adolescent needs seem to be met xdien stisdents are able to 
create a unique or original product, such as shewing "mood in a plaster 
mask" or using "orange coral and sea plants with orange fish" in a 
printit;g in an art class. Another form of participation that apparently 
is ei^ging is mental or physical manipulation, such as solving 
problems or applying concepts (e.g., in mathematics "there's lots of 
figuring, concepts to understand and apply"), or ej^jerjjoentir^ with 
equipment (e.g., "it was fun to work with the voltmeters and e55)eriroent 
with different voltages and li^t ba)l)s and stuff like that.") . 

Besides intrinsic interest and oppox amity to participate, five 
students reported that developing a better uixJerstandir^ of the 
material or gaining insists wc- their reason for being so engaged 
by a course. An exajiple of the former case is: "the class clarifies 
yhRt you Iiave alx^dy read at hom. It clears Mp a lot of confusion 
about it [cur political and econcmic system]." Aixi two students valued 
questions that "ej^x^se you to new ideas I'hat you'd never come across" 
and "writing exercises [vdiich] es^panded ny view of writing quite a 
bit.^' 

One of the reasons for the course being so engaging was attributed 
by approximately -Svo in five students to the teacher. A diverse range 
of reasons were given of vMch the most cosnmon concerned the teacher's 
instructional behavior. One student relayed hew her English teacher 
will "take a borii^ poem and point out things you'd never see and mate 
it interesting." Other students described the way their teacher 
e^^lained or structured the material, provided corrective feedbacik, and 
involved tie students by inviting them to discuss their e^^jeriences in 
relation to the topic being studied. In the former case, for example, 
one^ student was inpressed by the way his teacher ^'will start with like 
a piece of the question and work each piece togetner, and then ccme v ut 
with a broad answer, so that all of a sudden we will know it." Other 
teachers made a course interesting by telling jokes or having a sense 
of humor, relating interesting stories as examples, kidding around, ar^i 
generally making class fun. 

In adiltion to the relevance to daily life that made courses 
intrinsically interesting aixi worthvMle for many students, some 
students found a class engaging because of its relevance to their 
future goals. Statertvents such as "I'll use this knowledge after* 
sdiool," "in the future that's v4iere the jobs are going to be," arx3 "1 
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have to knew it and I'll be doing more of it later" are indicative of 
this instnimsntal form of relevrance. It is interesting to note that no 
students mentioned institutional rewards, such as good grades, as an 
e35)lanation for the most engaging course, and only one described a 
reason related to instrumental performance (e.g., the material was easy 
to learn) . 

Table 6C belcw lists in rank orxSer the most ccsnmon categories of 
reasons for \*ich (a) himanities (social studies and English) , and (b) 
all other subjects, were described as the most interesting and 
worthvMle school subjects that students had taken in the year of this 
stucfy. 

Table 6C: Most Oconm Stuaent Kc^pcts for Ncxninatiiig Humanities 

ard Other Subjects as ttei Most Engaging Class 
Rank ^ K iHtanities Rank All Other Subjects 

1 intrinsic interest in l intrinsic interest in 

subject mritter subject matter 

2 class discussions of 2 manipulating information 

opinions/ideas or things 

3 interestij>g/fun teacher 2 interesting/fun teacher 

Ihere is om notable difference between the reasons prwided, on 
the one hand, for social studies aM English classes being engaging, 
and^ on the other hand, for those mentior^ in relation to other 
subjects (i.e., science, mathertatics, foreign language, hi^ technology 
lab, art, health, and photography) . Although intrinsic interest 
subject laattsr vas the nost asminon reason in both cases, the second 
most frequent student ejiplanation of engagement in humanities classes 
was the opportunity to discuss one's own ideas or opinions or (in two 
cases) to hear those of others. In contrast, this reason was never 
Bcntioned in the case of other subjects. Insti*ad, one of the three most 
ccatanon reasons th^t non-hiManities subjects are engagirg is the 
oj^pof&mity to manipulate information or materials. 

For most students the appeal of discussions appears to be either 
the chance to merely get involved in the lesson (and ctjtain scsne relief 
fixra "the teacher only talking vMch gets boring"), or being able to 
ccfftpare (or check out the rtatus of) their own views with those of 
their peers. However, other reasons etKrged, such as a genuine interest 
or curiosity in hearing different opinions, and the stimulation of the 
ccetpetition involved in trying to "win" an argument against one's 
peers. And a coi^jle of students felt that discussions were more 
relaxing and easier because thsy demanded* less effort than other types 
of lessons, ihe first explanation is analagous to actively 
participating in other subjects throu^ problem-solving and "hands-on'^ 
activities. 

Given previous findings that social studies is not perceived by 
students to be relevant to life (Shau^messy & Halac^, 1985), the 
hi^ ranking of intrinsic interest - vMch half the students attributed 
to relevancy - is interes<;ing. Again, it suggests that the social 
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Studies teachers included in the HOT project sanple do differ frcsn 
typical teachers of this subject, at least In being able to make the 
subject matter intrinsically interesting. 

ExaBsAes and Attributes of an l &igacprrf fTTagg 

Each student was asted: Have you ever had a class or course vdiere 
you found the subject so interesting that time passed ouch more quicGcLy 
ttvm usual and ^lere you put forth your best effort? Every student 
arswered an the affirmative. Ihen they were aslcsd vSiat was the subject 
of this class. 5he frequency with vtddn each subject was mentioned is 
listed belcw in Table 6D. 



Table 6D: Subject of Rally Bigaging Class 







(n=45) 




Rank 




Students 


Subiect 


Frequency 


Ferjentaqe 


1 


Social Studies 


14 


31.1 


2 


Science 


11 


24.4 


3 


^thematics 


7 


15.6 


4 


English 


6 


13.3 


5 


Foreign language 


3 


6.7 


6 


OoBDpjter Science 


2 


4.4 


6 


Art 


2 


4.4 


8 


Tedmical Drawing 


_1 


2.2 






46* 


102.1* 



* One student rated two subjects equalli' engagir^. 

In ccmparison with the most engaging courses taken during the year 
an vAuch students were interviewed, social studies remained the most 
ppjlar goice f pr j really engaging, class, ever taken in hi^ school. 
— was Cx«=ea.y folluv^ by science v4iich was ncsninated by apprxaxiinatelv 
a quarter of the si?.Tdsnts. m fact, the number of students ncminatina 
scierice or mathematics (40%) was approximately the same as the nunib^ 
m^ionang humanities subjects (i.e., social studies and English) 



Tflhen askai ^ e>?)lain \*iat made the class or course so interesting 
and why time passed so quickly, more than four out of five studente 
gave a reason related to the intrinsic value of the subiect matter or 
activities in the class (see Table 6E) . Hie most frequent specific 
reason was again (as in the case of most engaging subject) that the 
si^Dect matter was intrinsically interesting. More than half of the 
stitots vSio selected a social studies class mentioned this reason, 
su^ as one viio found that "learning about the holocaust, tlia leaA 
system, and the 1787 Constitution was really interesting, and SDI 
Nicaragua and the contras - i really liked that." Biis student thei 
revealed v*iy she found these topics intrinsically interesting: "i 
wasn't sure about these things, like my mother and father talked about 
It [Nicaragua] a lot. [Now] I wa-ld talk to them about it and it was 
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really great [because] they'd bring up all these points." other subject 
matter (in social studies in four cases and one each in biology, 
chemistry and coirputer science) was apparently intrinsically 
interesting because of its relevance to real world f«-mts or to a 
student's interests and ooncerrs beyond school. Exanples frcm social 
studies included: "some of the material was related to ny father's law 
practice," world cultures is very interesting because "iry mother is 
with the airlines and we travel a lot," and "she got us into it 
[genetics] because it was about us ... lite the reasons vhv I have 
black hair or brown eyes." 

Science was another subject that a number of students r^rted as 
intrinsically interesting: f^r exaitple, "there's so mani^ unbelievable 
things that just rattle ray mind, lite how little tiny organisms are 
found and hew things are created. " 

Scane students indicated that a class was particularly engaging 
because of the opportunity to manipulate information or materials. 
Nearly half these cases were mentioned in relation to mathematics, but 
this reason also was given for social studies, and "hands-on" 
activities in science and art. Two descriptions given for mathematics 
were "graphing a polyncenial function is intere^sting because yaa can do 
so much with it," and "I lite to figure out things \diere you have to 
work out or fix a problem." ,anoti-ier student described how in art class 
it "excited" her as the shape and form of a ceramic piece changed \^en 
she placed it in a kiln. 

O/er a third attributed the reason for a being particularly 

interesting and worthwhile to the teaciier. kcesentii^ a variety of 
activities or tcpics; and providing effective instruction (e.g., 
challenging- and clearly structured taste) were the most popular teacher 
behaviors, although nine different types were described. Other teacher 
behaviors mentioned by se'/eral students included giving us "fun 
activities" (examples of which are described in the next section) 
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TatUja 6E 

Beasons a Itoticular Ocaursa oar Class was Errtaodncf 

(n=45) 



instruL.antal 
perf onnance in 
class 



1 2.2 naterial was easy to learn 
1 2.2 performed better than others 



instrumental 
value of class 

intrinsic 
perf onnance in 
Claris 2«2% 

intrinsic 
value of class 
82*2% 



teacher 
b^viors 
37.8% 



peer 

behaviors 
6.7% 



6 
6 



6 



4 

4 

3 
3 



2.2 perfonned v^l 



18 40.0 



20.0 



intrinsically interesting subject 
matter 

opportunity to manipulate 
information or materials 
13.3 cognitively cshallenging tasks 
13.3 opportunity for discussion or 

contribution of cwn ideas/opinions 
opportunity to create a unique or 
original product 
13.3 learned new things or gained new 
insists 

cfpportunity to hear ideas/opinions 

of others 
opportunity to i^vise previous 
material 



13.3 



6.7 



2.2 



8 17.8 



instructional practices: made 
subject interesting (fun 
activities, stories), challenging 
personal characteristics (likeable, 
enthusiastic, sense of humor) 
8.9 provided variety (of activities or 
topics) 

knowledgeable aho?it subject 
attitude tcwards students (caring, 
gave responsibility) 



8.9 



6.7 
6.7 



2 4.4 students in class were all friends 
1 2.2 serious attitude to learning 
1 2.2 acadomically coarpetitive 



* Numbers add to more; tlrian 45 and percentages ac2d to more than 100 ^ -3 
more than one reason could apply to any response. 
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being knowledgeable about the subject, and having a personable or 
li k eable disposition. 

An instmnental reason was given by only two students, and a 
reason associated with the behavior of their peers was mentioned bv 
only four students. 

Rjllowing the discussion of the particuxar class or course that 
each_ student selected as really engaging, a five-point scale was 
provided on \vtj-.ch the student indicated how often he or she had been 
involved in sjch an engaging class throu^out high school. .Students 
were evenly split between the those ^o said fairly or very often, and 
those v^o reported only e^^jeriencing such engagement saraetir.ies (the 
maDority) , once in a vfeLle or (for one student) practically never. 

Mast Enqacriicr Activities or Tasks 

Students were asked to describe some of the most interesting 
activities they did in the course viiere the subject was so interesting 
that time passed much more quickly than usual and vdiere they put forth 
their best effort, six students either could not recall a specific 
a±Ea^cr2ddea«ri:ttBe\eeroptr^egglgactivities. When the descriptions of 
the remaining 39 students were categorized according to the four 
dimensional coding scheme (see Table 6F) , there was only one dimension 
on which a majority described the same characteristic and that was on 
interaction v^ere approximately sixty percent indicated an activity 
done alone. One-third of the students mentioned a writing task, but the 
remainder described seven of the ei^t other activities or tasks with 
only one, an e^^periment, being reported by more than ten percent. 

Unfortunately, there was a iiigh frequency (one in four) of 
uncodeable responses on the content dimension because many descriptions 
of engaging activities did not include details of the specific content 
fT*^^^^?^!!^^®'^ directed to prdba for these details. Almost 

half of the codeable responses (or just over one third of all stucW 
responses) involved an activity or task in vMch the content was 
concerned with the principal ideas or central meaning of the material 
being studied. Some exaitples of such activities were: 

-learning about symbolism and applying it to our own reading, 

-doing research in the library for an essay on '"Ihe Federalist era: 
Did domestic and foreign policy endanger or secure the nation?" and 
tt^n "seeing how historians view the same time period differently", 

-"The teacher demonstrated an experiment with a cartesian diver 
vhich sank v*ien it was squeezed. We had to figure out v^y this 
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Table 6F 
Frequencies and Bgroentaqes* of 
Staadents' Descriptions of Activities in an Encocfim Class 





no. 


percent 


writing task 


15 


34.1 


«5:ariment 


6 


13.6 


reading (only) task 


4 


9.1 


discussion 


4 


£.1 


debate, role-play 
rieia Tzrip 


4 


9.1 


3 


6.8 


film 


3 


6.8 


oral presentation 


2 


4.5 


vmcodeable 


_3 


6.8 




44*4; 


99.9 


interaction 


no. 


percent 


self 


26 


59.1 


peers 


10 


22.7 


ceacner 


5 


11.4 


uncodeable 


_3 


6.8 




44** 


100.0 


content 


no. 


percent 


ideas 


17 


35.4 


esqperiences 


8 


16.7 


values 


4 


8.3 


specific topic 


4 


8.3 


facts 


2 


4.2 


uncodeable 


13 


27.0 




48*** 


99.9 


function 


no. 


percent 


manipulation 


21 


41.2 


intake 


13 


25.5 


creation 


7 


13.7 


demonstration 


3 


5.9 


uncodeable 


_7 


13.7 




51*** 


100.0 



* Of total number of re^nses on each dimension from 39 students vrtio 
described an engaging lesson or assignment. 
** Five students described two activities. 

*** Mbre than one category on this dimension could ajply to any one 
response. 
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happened. I spent hours and hours trying to figure it out, 
althou^ it was only worth a coiple of extra credit points for 
which you norroally wouldn't spend too much tiitve." 

Sam students described an activity concerned with vinderstanding 
the e}<periences, feelings or emotions of oneself or others. Often this 
involved trying to understand the perspective of people vftio lived in a 
different time or culture, such as '»inovies which make a culture live by 
showing you the way of life and giving you a feel for the culture." And 
scapet2JD(ves it involved journal writing "to eicpress your views" on a 
painting, event or some other personal es^^erience. 

Given that almost one quarter of the i-rtrudents nominated their most 
engaging course (in the current school year) because of the o>7portunity 
it provided to contribute their cpinions or ideas, it was somei^iiat 
surprising that less than 10 percent discussed an activity viiere va].ue 
judgements had to be made or examined. Instead the activities described 
by these students ranged across all types of content. 

Only two students mentioned an activity enphasizing the learning 
of discrete facts, but in both cases a hi^ier level cognitive activity 
also was involved. For exaitple, one student discussed doing a map of 
the eastern hemisphere on \Mch "we had to name the countries", but 
added that they also were required to e)5)lain "vdiy th^ had alliances 
with their different countries." Ihis finding suggests that students do 
not find activities engaging vtoen the content involves only isolated 
facts or fragmented bitf of information. 

In terms of the purpose of the interesting activities described by 
students, maiy of the codeable exaiiples were intended apparently to 
have students manipulate information in order to make sense of it. Such 
activities included solving word problems and equations, organizing and 
synthesizing arguments for a debate on Indian land ri^^ts, analyzing 
information to crces-examine a witness in a simulated court trial, 
conducting a survey on an environmental problem, and formulating 
questions on an oceanography experiment for another gro^ to answer. 
Activities concerned only with acquiring information represented one in 
four responses. Examples of these intake acti.vities were: viewing 
sliders on Spanish speaking countries, reading different books like All 
Quiet on the Western Fronts listening to the teacher e>5)lain symbolism 
and to other students give their qpinions on evolution, and going to an 
"art show for critical appreciation." A number of students mentioned 
activities that encouraged them to use newly acquired knowledge or 
skills to create an original product, such as developing a conpiter 
program, making a poster witii a metaphorical representation of a 
concept, and writing original essays or reports. 

Across the four dimensions, an individual writing task requiring 
the manipulation of ideas was the single most often described task, 
representing 20 percent of the responses. This finding contrasts with 
reports of teachers* cnnplaints about student resistance to substantive 
academic tasks (Hampel, 1986; McNeil, 1986; Tye, 1985). 
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Enqaqiiiq Socdal st ^i^^j^ TAg sms and Assignments 

Students v;ere asked if they had ejqperienced a lesson or an 
assignment in the social studies cairse deserved by a researcher vMch 
they found so intei?iisting that tiitve passed much itore quickly than 
usual, and \^*iere they made nore effort than they nontally do in school 
and really tried to do their best. All but ei^t students indicated 
that they had. Ihese students then were asked to describe v*iat the 
lesson or assignment involved. 

The categorization of the responses on the four diraensional coding 
scheme (see T&ble 6G) resulted in a sonewhat similar frequency 
configuration to that for the descriptions of activities in a 
particularly engaging class. A writing task again was the inost often 
mentioned of task (by almost a third) , but in this instance was 
closely followed by discussion, with debate/role-play also reasonably 
popular. These last two activities c jnerally occur more ftequently in 
social studies (and English) than in other subjects and hence their 
hi^er frequencies in a social studies context are not surprising. 
Consistent with the pc^^ularity of writing assignments, individual tasks 
were more ccramonly described than activities or tasks involving 
interactions with peers or the teacher. 

It is interesting to note that the vast majority of these 
individual tasks (or approximately one third of the engagir^ exaitples 
students than class activities, especially those enabling interactions 
with peers. 

Ideas and values dcmdnated the content of students' examples of 
engaging lessons or assignments In the HOT social studies course. Ideas 
were involved in such subject matter as the law of simply and demand, 
hypotheses about hunan behavior, symbols and images associated with 
Andrew Jackson's personification of American traits, the motives of the 
founding fathers (mentioned by three students) , and interpr^tions of 
the Federalist era and the presidencies of George Washington and Dicmas 
Jefferson. Value issues included such exanples as deciding: viiether the 
^^Aiiskey rd^ellion was justified, whether immigration was good for the 
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TatOA 66 
Frequencies and Peacoentaaes* far 
Students'DescxdpbioTs of Lessens and Tasks in 
Social Studies Oourse that are Enoacrinj 
(rF=37) 



type 


no. 


percent 


variting task. 


13 


31.7 


discaission 


10 


24.4 


debate, role-play 
reading (only) task 


7 


17.1 


2 


4.9 


questioning 


2 


4.9 


oral presentation 


2 


4.9 


film 


2 


4.9 


uncodeable 


3 


7.3 




41** 


100.1 


interaction 


no. 


percent 


self 


17 


41.5 


peers 


10 


24.4 


teadier 


10 


24.4 


uncodeable 




9.8 




41** 


100.1 


content 


no. 


percent 


Ideas 


17 


41.5 


values 


12 


29.3 


specinc uopic 


5 


12.2 


ej^jeriences 


2 


4.9 


facts 


2 


4.9 


uncodeable 


_3 


.7.3 




41** 


100.1 


function 


no. 


percent 


manipulation 


22 


53.7 


intake 


8 


19.5 


creation 


8 


19.5 


demonstration 


0 


0 


uncodeable 


_3 


7.3 




41** 


100.0 



* Of tota]. number of responses on each dbnension fran 37 students vdio 
described an er^gii^ lesson or assignment. 

** Four students described t/ra activities from les^son/assignment. 
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country, viiat were the characteristics of a good congressman or 
senator, \^ther a nuclear freeze should be advocated (the exanple 
provided by all three students frcstn one class), and ^ther there 
should be stricter gun control laws. 

The function of the majority of lessons and tasks described by 
students was to manipulate or make sense of information, usually by 
analyzing ideas or evaluating issues such as those listed above* 
Wtiting^ a book critique, deciding one's position and developing 
supporting arguments for a oLass debate or essay, and constructing a 
visual and descriptive metaphor were examples of the kind of r^rted 
tasks whose primary purpose matched this categc^i^. In only one case was 
an example given of a task whose function essentially seened to be 
enabling students to demonstrate the knowledge (or skills) they had 
acquired - and even here the student criticized that aspect of the 
task: 



•Tfe did a written report on Jackson's personification of American 
traits. It xtfas sort of stipid because all we did was put ideas in 
the book into our own words. We had to describe two or three 
specific images of America that Jackson was a symbol of. It was a 
200 page book, but it didn't take that long to read becaxase I found 

it really interesting It was irore interesting because you're 

dealing with symbols and images. One image was that Jackson thou^t 

that God was on his side I did have to put the book in my own 

words, but ttie subject was mors interesting." 

Iha-a findings suggest that generally these students seem to be 
engaged by social studies tasks vMch allow them to actively ej^lore 
various ideas and values. But to understand this preference more fully, 
we need to examine their reasons for finding these lessons or t^ks 
engaging. 

As Table 6H indicates, the vast majority of students reported that 
the lesson or assignment they had described was so interesting because 
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Iteag< 3>^ '^'^l s^v^^Qg T#>ggfyi car T&slc was Brps^ m 
(n=35) 



instnmental 
valiie of task 



intrinsic 
performance 
on task 
5.7% 



5.7 perforroed well or did a good job 



intrinsic 
value of task 
82.9% 



14 40.0 intrinsically interesting subject 
mtter 

10 28.6 opportunity for discussjion or 

contribution of own ideas/cpinions 
9 25.7 learned new thii^s or gained new 
insists 

5 14.3 cognitively challenging task 

5 14.3 oj^rtunity to hear ideas/qpinions 

of others 
3 8.6 ofpjportunity to manipulate 

information or things 
2 5.7 made historical event or figure 

real 

2 5.7 opportunity to play unique role 



teacher 
behaviors 
14.3% 



3 8.6 knowledgeable about subject 

2 5.7 instructional practices 

1 2.9 personal characteristics (likeable) 



peer 

behaviors 
2.9% 



2.9 cooperative approach to work 



uncodeable 



2.9 



* Numbers add to more than 35 and percentages add to more than 100 
more than one reason could apply to any response. 
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of the intrinsic value of the task. Ihree specific reasons within this 
general category were frequently mentioned. Ihe first, and roost cannon, 
was that the subject natter was intrinsically interestir^, iMcii 
included such topics as the theory' of natural selection, ancient Greek 
civilization, criitdnal trials, women's ri^ts, and the Constitutioii. 
For scxne students it was a particular fact on care of these topics that 
was engaging: "I found it fascinating . . how we have been on earth for 
such a sn>all fraction of the li^Je of earth," and "you wanted to know 
the answer as to vSr/ this mi^t be a fact because fact itself [i.e., 
people who drive white cars have a itiuch hi^ier rate of skin cancer] was 
so interesting." other students, hcrjevesr, had a roore conplex 
e}?)lanation of vdiy they found a topic intrinsically interesting, for 
exanple: 

"I liked writing about it [v*iether the founding fathers were 
trying to perpetuate their own class interests], it's 
interesting because it goes back to Marx and class analysis. 
Taking economics got me interested in Marx and my father has 
been talking about Marx for years. This gave me a chance to 
apply critical views that are otherwise put down by students 
vdic l-iave a Raitibo-like mentality." 

The above quotation also illustrates the second r^^rted reason in 
tiie category of intrinsic task value, the opportunity to contribute 
one's own ideas. A role play of a debate at the Constitutional 
Convention provided such an cppoj±unity for another student by enabling 
him, in his words, "to spe^ your mind and get across your ideas" 
th-rou^ a particular character. 

learning something new or gaining a better insist represented the 
third common reason. Typical statements were: "I never raized before 
vdiat actually happens" (in the process of arraignment), "I hadn't 
encountered it [the revisionist view] before, vdiereas everythina else 
in American history I have had before," and: 

"m the first few days of this class the teacher put im the 
quotation: 'Where you stand depends on where you sit. ' I found that 
interesting because it presents you v.Lth a different vxi mare 
realistic side of American history. It was something ne- -a I had 
never looked at it that way before." 

Again these last two quotations indicate an interest in ideas or 
world views, rather than factual information (as in the first 
statement) . Interestingly, no instrumental reasons were stated and only 
a few students directly attributed their engagement in a lesson or ta^ 
to their teacher. 
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Enqacrincr Questions or Tasks in (fcserved Social StucLLes Lessogis 

As well as the questions about school subjects in general and the 
social studies course in particular, students eilso were questioned on 
each of the three visits about a social studies lesson observed by a 
iBseardher. Ihey were asked vdiat, if anything, did they find 
inteiBsting about that le:Sson and v^^t made it interesting for them? 

©le responses were categorized on two dimensions: one involving 
the cognitive type of question or task, and the other the intended 
function or cognitive purpose (see Table 61) . In the latter case^, half 
the responses coild be categorized within the function of intake) that 
is they involve situaticns vdiere the student was acquiring 
informatiai, usually by listening to the teacher or to other students' 
answers to a question or task. The other half of the responses 
described questions or tasks \*iere the student had to engage in active 
inental effort to make sense of infontaticai, or in a few instancefs 
create a vmique product, in order to respond to a question or task. 
Scsne exaitples of manipulative function questions were: determining '*why 
'"'^ leaders are better than others," arguing vdiether civil 
oedience is justified, '-trying to find out vdiether teciinology was 
i*^uving the government spend more or less - we really had to think about 
it," and "trying to list the similarities of the three industrial 
tycoons." An exanple of creation was a small group task requiring the 
creation of a court case involving First Amendment ri^ts. 

Ihe most common cognitive^ type cf ques tion or task (described by 
almost half the students) concemecl the relations between facts, such 
as conparisons and contrasts among different facts, aid explanations 
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Table 61 

DascriiJtiaa cf EnaaaiiiCT Oasstic re; or Tasks 
in Obs erved .^nnial 5^irii^ T<:>gflons 
(1^=105) 



ooqnitive type no. percent * 

relaticans between facts 48 45.7 

- required cccprehension only -22 - 21.0 

- required inference - 15 - 14.3 

- required abstraction - b - 7.6 

• required eirpathy - 3 - 2.9 
evaliaation 24 22.8 

• justifications of opinions - 14 - 13.3 

- cjpinions - 10 - 9.5 
specific facts or generalizations 22 21.0 
definitions 1 1,0 

uncodeable _ig 9,5 

105 100.0 

function 

iiitake 54 51.4 

manipulation 46 43.8 

creation 4 3.8 

tmcodeable 1 1,0 

105 100.0 



* Percentages computed by dividing the number of student responsa<5 
in each category by the total number of responses (i.e., 105) . 

Note: 1. In 14 cases students indicated there were no engaging 
questions or tasks in an dbser/ed lesson. 

2. In six cases students reported that the lesson in general was 
engaging, but could not think of a specific question or task. 

3. Twelve students were not takirig the saoe social studies course 
(i.e. , the one included in the HOT sample) on the third visit and h*?nce 
were questioned on only two observed lessons. 

4. Ihree students were absent from the observed lessons on one 
visi'c and hence were questioned on only 'Irwo le'ssons. 

5. Five students described two engagii^ questions trm an 
observed lesson. 
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(Of event?? or states) giving causes, ccnssqusncas or predictions 
(Hyman, 1979) . If these cCitparisons and e}$)lanations were made for the 
student v^o luerely acguired the infoniation (that is, the function was 
intake) , then the rasponse was categorized as carprehei^ '.on. Fourty 
percent of intalce responses were conprehension of relations beta-.-een 
facts. On t>>e aCrsa: hand, if the student actively participated in 
constructiiig an answer ty interpreting, analyzing or itanipulating 
information., then the description was categorized as denonding 
inference, abstraction, or enpathy, using Newmann's (in press) 
delineations of content challenges in social studies. For exanple, the 
question (posed by a student) of "How did America avoid a revolution at 
the tiine,. given the extremely bad working conditicns arri the pronouncea 
s^aration between the classes in terms of wealth?" represented an 
inferential t^/Jx. An exairple of abstraction was interpreting econcmic 
data and ej?>laining how increasing the number of workers can eventually 
decrease revenue. And one of only three examples of empathy involved a 
role play of the Constitutional Convention in ^ch a student had to 
try to understand and argue the position of her character in si^porting 
both slavery and a Constitution intended to protect people's ri^ts. 

Evaluation questions or tasks (cited L almost one in four) 
involvai either the justification of an opinion or value judgement, or 
merely the expression of an opinion or making of a value judgement. 
B^laining one's opinion on 'Mhat would yo-Ji look for in a democratic 
leader?", and "defending the ri^ts of the American Indians, that they 
had a moral argument to keep the land" wera two responses in the first 
category. While most of the responses in the second category described 
making personal value judgements, a few responses emphasized hearing 
the pinions of peers and conparing them to the student's own views on 
a particular question. For exaitple, one student stated her interest in 
seeing how the class classified different countries on various 
adjectives and in wondering "vdiat other people are thinking of when 
they choose something different to you." Another student liked fii>ding 
out people's views on, for exaitple. South Africa so that he could 
"learn vdio I can talk with" and how "I can tailor my responses to get 
them to respond." ^ 

With only one exception, questions or tasks involving specific 
facts or generalizations involved an intake, or acouisition of 
information, function. A typical example was "finding cut how we got 
our stat^ of California and Texas - because America went to war for 
those states." 

Questions intended to elicit an empiricaJ response (that is, 
demanded facts, e}?)lanations or conclusions based on facts, or 
inferences drawn from facts) were cited in over 70 percent the 
codeable responses. Ihis result, however, may not necessarily represent 
the kinds of questions students find tha most engaging, but reflect the 
damnnant mode of questions that students face. Althou^ we have no data 
on Jie respective percentages of question types posed during these 
observed lessons, a stuc^ of the teaching of a social studiS toiDic 
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rspcrtsS that c^rsr 30 percent of the questions jftere of the facts or 
relations between facts type (Bellack et al, 1966) . 

Encgwement in Ih-depth Wtaric 

Cbncems have been e>55ressed about the predaninant eT5:hasis on 
cxverage of content in hi^ school classrocsns and the negative ef f feet 
of this enjiiasis on student engagament (Newmann, 1988) . therefore, it 
seemed ijseful to f iid out vSiether or not students fouid the alternative 
to a coverage aKaroach, that is in-depth wrk, ei^iging. Students were 
asked if tl^ had ever had a class or course in ^>duch they Lpent a 
considerable amount of time (say, two or more wseks) digging de^ly 
into a specific t<^ic, question or problem, Aliaost two-thirxJs (64.4%) 
replied that they had. VBien asked in v*iat class this e35)erience 
occurred, i^arly all responses vjere either social studies (14) or 
Fnglisli (13) , with science (4) being the only other subject rvsntioned 
by more than one student. 

Ihose students v*io had e55)erienced in-^pth work were asked if 
there was anything they found interesting or worttaMle about this 
work. Only one student said there was nothing interesting or 
worthvMle, and three indicated that it depended on the topic or tbr 
aitount of time (e.g., '•beyond five days it either becomes borii>g 
frustrating as you want to change your whole essay becax:ise you becon 
confused and charge your opinion on the essay question.") . 

Students generally stated that in-depth work was interesting 
because they liked the pai'lcular subject or t^ic, or because it 
provided the oj^rtunity for active participation that enablea them to 
be creative or to work autonomously, sucih as doing their own research 
for a paper. For some students interest in or liking for the topic 
evolved fcm the process of in-depth study: 

"If you takp one subject that you don^t i2n3erstand at all, or don't 
"hhink would in'.juiTist you, and then over the course oi two weeks^ 
whatever, for me I jiost fall in love with tiiat topic. ... It got to the 
point that this book [on the Red Baron filter pilc?t] was part of my 
life... I wasn't nervous [in giving an oral report to the class] about 
anything, I didn* need notes, it was like all scarred in my head." 

Similarly, other students indicated that their engagement resulted 
from active participation in the work. For exairple, one student 
r^rted "I had a lot of research to do. Doing things on my own ... 
goirg to the library and looking things up made it interestirjg for me", 
and another said "as I did the research I got irvore interested in it." 
Yet another student described a creative writing assignn^nt in ^ch 
her engagertent stenined frm "coming \j(p with an idea and a plot that I 
thou^t was gr*^t.'' 

It was the worttodiileness of in-depth work that was en^iasized by 
many students. As the following two exairples illiistrate, the result of 
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examiiimg a tcpic in detail can be painless learning and enhanced 
understanding. 

"You Isamel a lot. We did a project on the praying mantis. We took 
pictures, went to tlie library, took lots of notes, made a 
billboard, and pr^ssented it to the class. ... You didn't realize 
that you were really learning. That's worthvMle!" 

"Wis spent three weeiks devote entirely to the constitutional 
convention. It was an incredible experter.ce because you could 
really get the feelings of the events and share with the people 
vdiat happened and " 

Itdbably the most can^irlling testimony for in-depth work, rather 
than an enfiiasis on cwerage, was provided by a grade 11 studentt 

"I got totally immersed in a project vhen the teacher forced us to 
do a parer on sane gay. We couldn't pick him, but we had to read at 
least four books and write at least 100 note cards (big cards) , and 
develop at least a 10 page paper. I got Montaigne, it ended v© real 
interesting. As Mr. IXjrHey pointed out, it was kind of cool that I 
got to be a real ej^jert and to knew more about this 9W than 
probably five million people in America. I'm not sure what made it 
so interesting - vSiether it was Montaigne's own works and life or 
■just the fact that I got to know so much about him. Most of the 
tune, you don't get this in school. A lot of times it's a total 
skjn; It's very bad. A classic exaitple is this course in European 
history. We covered 2000 years. Every week we had a 30 page 
chapter due. it's one of the hardest courses in the school. A real 
lot of work. He's a stickler for dates, all dates ai^ the facts. We 
had 50 dates a week to memorize. The pity of it all is that now I 
don't remember anything. I worked so harS, and now basically all I 
remember is Montaigne. There's like mybe five dates I remember 
x^en I probably learned tliree or four hundred dates all year, i 
can't even remember even a lot of the major guys we studied." 

Despite u number of such powerful eiqjlanations of the value of in- 
depth work, the students were almost evenly divided between those vho 
would li^ to havrt more of this type of work in school (15) and those 
vdio would not (12) . Among those v^o did not want itore in-depth work, 
several felt that it involved too much work even thou^ it was 
vrorthvfliile (e.g., "I'm lazy", "Because it was so hard"), v*iile some 
preferred either to obtain a broad knowledge of a subject ("If you did 
that all year you wouldn't get to cover other topics.") or to have move 
variety ("I like \-ariety with new stuff each day."). The stJdent cited 
above v^o did the project on Montaigne wanted more in-depth studv, but 
recognized the dilemsta of concentrating on broad coverage or depth of 
knowledge: 

"Yes, I'd like to have more work where you dig in depth, but it's a 
double-edged sword, 'cause if you're constantly goii^ in depth 
about each thitjg you come across, then you're not goino to get very 
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far. It^s qoantily versus quality. The only reasonable tiling is 
you've got to find a balance. I guess there's iiore of the 
si?>erficial quantity in school now teachers trying to cover as 
much as they can. They're not going really into depth." 

Teacher Qiaracteristics asscciated vith Eroaqemeait 

Having explained vAxy a particular class was so interesting that 
tiite passed much wore quickly than visual (and described sosne of the 
Host engaging activities in that class) , students also v/ere asked \«*iat 
stands out in their mind about the teacher. Four students indicated 
nothing, but as Tfeible 6J reveals, the rest described a diverse range of 
c^iaracteristics. Various instructional practices, particailar attitudes 
tcwards students, and a nuiribe. of personal characteristics were most 
often mentioned. 

Making the subject interesting Isy providing fun or engaging 
activrties, relating interesting stories or anecdotes, or in sorae other 
unspecified way was the most camrtonly described teacher behavior. One 
student said her Spanish teacher "teaches in a normal way but adds 
things that are fun: for exaiTi)le, playii^ gaites to learn or review for 
a test, singing sor^, teaching the different meanings of certain 
words." Othtr teachers made the class interesting because they "made 
like, the most boring stories esajiting", such as one vdio "would really 
get into it . .he was like kind of a kid . . .he would tell us all these 
interesting stories" and another ^lo "remembers what he disliked ai^t 
chemistry and tries to mr'ke that interesting x>y. for example, telling 
these bewildering facts: for eranple, about how one mole of rice would 
cover the entire world 60 meters deep, and then he e>?)lained bow he got 
this figure." 

A number of other general instractional practices made the teacher 
stand out for a few students in each case. These were: eiKJOuraging 
student participation in the class (e.g., "he gives students a lot of 
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OJaible 6J 

Pesc ripfcions of Tieachers in Enoaciincy Class 
■ (n=41) 

no. % 



knowledge 
17.1% 



instructional 
practices 
61.0% 



attitude 
towards 
students 
46.3% 



personal 
character- 
istics 
43.9% 



academic 
expectations 
7.3% 



5 
2 

2 

1 



4 

4 
3 
2 

2 



7 
4 
3 
1 



8 
7 
4 
3 



2 
1 



12.2 knowledgeable aboiit subject 
4.9 eiqjerientially knowledgeable about 
subject 

4.9 knowledgeable about adolescent learning 

needs and interests 
2.4 knowledgeaJole about a variety of topics 

or issues 



13 31.7 makes subject interesting (e.g. fun 
activities, stories, anecdotes) 
enwurages student participation 
oihailenges students' thinking 
gives lots of exairples 
explains material at appropriate level 
gives well-organized lessons, clear 
tasks 

gives helpful feedback 



9.8 
9.8 
7.3 
4-9 
4.9 



1 2.4" 
8 19.5 



17.1 
9.P 
7.3 
2.4 

2.4 

19.5 
17.1 
9.8 
7.3 

2.4 



4.9 
2.4 



concerned about helping students 

xmderstand the material 
gives students freedcaVre^nsibility 
respects students' opinions, ideas 
fair, treats all students equally 
encourages and praises students' 

efforts 

shows genuine interest in adolescents 

sense of humor, teJls jokes, etc. 
firiendly, likeable person 
enthusicistic about teaching subject 
open-mirx3ed, accepts new ideas or 

presents all sides of an argument 
confiderit, articulate, warm spea3dng 

sti'le 

strict demands for ccirpleting vrork 
expects quality work, demands student's 
best effort 



* Numbers add to more than 41 ard percentages add to more than 100 as 
more than one category of teacher characteristics could apply to any 
response. Percentages were computed by di: "aing the number of students 
responding in each category by the total number of students (i.e., 41) . 
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chanoe to talk, discuss and debate things"), challenging students' 
thinking (e.g., "he asks a lot of questions that you have to think 
about"), providing lots of exairples ("to make it understandable''), 
e>5)laining the material at the students' level (e.g., "she breaks down 
tou^ words into your language instead of stating tJiem in scientific 
terms") , ard giving well-organized lessons or providing clearly defined 
tasks (e.g., "she makes class flew easily - thing ties into 
another, connections between units") . 

A grocqp of attitudes towards students that couiu be characterized 
as reflectiiig a ccsnbination of a caring cone yrn and an adult-like 
respect was mentioned by almost one in twcrespondenfcs. The most 
frequently discussed was a demonstrated concern for helping students to 
leani and xanderstand the material being studied in class. Ocaratents such 
as "she didn't get frustrated \*ien I was getting the wrong answers and 
stuff and she helped me out a lot," and "she spends time on irrportant 
things: asking about what you didn't vmderstand" were typical of this 
category. Almost as many students identified their teacher ' s 
willingness to give them a certain amount of freedom and responsibility 
as a significant attribute. Examples students gave of this attitude 
v/ere: 'Ttie trusts kids, for exairp3.e with bunsen burners, and thinks we 
are responsible," "she allows a lot of freedom (for exairple, you can 
sit with your friends, you can bring radios into class)," and "she 
gives them a chance to do vdiat they want." Shewing respect for 
students' opinions and ideas (e.g., "she incorporated kids' ideas into 
the class,'" "he is polite and he accepts vdiat you have to say even if 
soroetiities you are wrong about something, he still respects your 
opinion") and treating all students fairly and equally (e.g., "he 
didn't favor any individual and gave everybocfy his time") were each 
reported by several students to be characteristics of their teacher 
that stood out. 

Such personal characteristics as a sense of humor, a likeable or 
friendJy personality, enthusiasm for teacMng, and qpen-mindedness also 
were described by nearly one in two students. A teacher's willingness 
to occasionally be funny, tell jokes, or just generally e:jdubit a sense 
of humor was iitportant to a number of students ir making a class 
engaging, vMle being friendly or nice \^;as engaging to others (e.g., 
"he keeps a friendly relatioriship with the students, all tlie class 
basically likes him") . A few students felt that being enthusiastic 
about and energetic in teaching (e.g., "she's into it, loves teaching") 
was an ouistanding characteristic of their teacher; vMle sai^ others 
designated having an open mind (manifested, for exairple, by a readiness 
to "accept new theories") . 

Only a relatively small ramiber of students indicated that the 
teacher stood out because he or she was knowledgeable. Besides the 
predictable attribute of being know2.edgeable about one's subject, which 
was mentioned surprisingly by only a handful, being 3aiowledgeable: about 
adolescents ("he knows vtoat kids like and dislike") and about a variety 
of other subjects or issues ("you can bring out anything and siie talk 
knowledgeably about it") were other categories described by a caaple of 
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sb^ents. And only three students mentioned academic expectations as a 
feature of the teacher in their particularly engaging class that stood 
out in their mind, ihis last finding is particularly inportant given 
the eitjiiasis in the effective schools literature on teachers' hi^ 
expectations of all students (Purkey & Smith, 1983) . 

m addition to discussing the teacher of a particularly engaging 
class, students were questioned more generally about are/ teachers v^o 
stand out because they push them to put forth the.ir best effort. TWo 
questions were posed to the students (all but two) v*io indicated there 
were such teachers: How do they get the best froo you? How do these 
teachers differ from others? Responses to these two questions were 
treaty as one re^nse and analyzed together since the second question 
essentially was askiiig students to elaborate on their first response by 
comparing the teacher(s) to others. Again instructional practices and 
attitudes _ towards students were the most frequently designated 
characteristics, but this time as mair^ students mentioned academic 
expectations as personal ; iiaracteristics (see Table 6K) . 

A concern for helping students understand the material was easily 
the most ccanmon response. Typical statements included "they'll give you 
help if you need it" and "they show they care if you learn," while 
"other teachers don't try to help you along with a problem, they just 
give it to you." A more unusual but perceptive comment was "they don't 
give you busy work, but just enough for you to understand the 
material." Some students also erojphasized the teacher's patience arA 
individual attention, as the following exairples illustrate: 

"Well, he ej^jlains wrk to you and if you don't know and you try 
and do it and you still can't, he still helps you and doesn't get 
frustrated." 

"These teadiers take tiitie with you, they go over it ags^.i and 
again. They have patience." 

Encouraging and praising students' efforts and showing a genuine 
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Tahle 6K 

Descxipticns of Teadiers ^^to Push SbaSaits to Best Effort 

(nF43) 

no.* %* 



knowledge 
7.0% 



instructional 
prac±ices 
58.1% 



attitude 
towards 
students 
60.5% 



personal 
diaracter- 
istics 
37.2% 



academic 
eJ5)ectations 
39.5% 



3 
1 



8 
8 

6 
4 
3 
2 



7.0 
2.3 



4 
3 
3 



4 

1 



18.6 
18.6 

14.0 
9.3 
7.0 
4.7 



18 41.9 



7 16.3 



5 11.6 



9.3 
7.0 
7.0 



10 23.3 
5 11.6 



9.3 
2.3 



6 14.0 

5 11.6 

4 9.3 

1 2.3 

1 2.3 



taiowledgeable about subject content 
taiowledgeable about adolescent rv^eds 
and capacity for learning 

encourages student participation 
makes subject interesting (e.g. fun 

activities, stories, anecdotes) 
challenges students' thirfdng 
es^lains material at appropriate^, level 
gives helpful feedback 
gives well-organized lessons, clear 

tasks 

concerr^ about helping students 

understand the material 
encourages and praises students' 

efforts 

shows genuine interest in and 

caring concern for adolescents 
gives studants freedat/responsibility 
fair, treats all students equally 
respects students' opinions, ideas 

friendly, likeable person 
enthusiastic about teaching subject 
sense of humor, tells jokes, etc. 
confident, articulate, warm speaking 
style 

strict demands for ccirpleting work 
assigns and requires a lot of work 
want you- to-work and do mil 

without making strict demands 
e>55ects qualily work, demands student's 

best effort 

uses embarrassment or threats of 
punishment 



* Nwribers add to more than 43 and percentages add to n-ore than 100 as 
more than one category of teacher charact-sristics could apply to an^ 
response. Percentages were costputed by dividing the number of students 
responding in each category by the total number of students (i.e., 43) . 
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interest in students' lives were two other ways in vMch identified 
teachers demonstrated a caring concern, accorxiirig to a nurtiber of 
students. Exairples of the fomer category included "£he tells ms that I 
can do it if I ji:ist keep trying" and "he keeps tellir^ ine how great the 
defense raechanisnts assignment was, [and] makes you feel good about the 
way you think about something." Representative of the latter category 
were the comments: "she became a friend vftiom you can talk to about 
anything," "he knows you as an individual? if you ccane xjp to him after 
class he remembers about your personal life and past stories," but 
"other teachers don't seem to like kids or v4iat they're doing: it's sad 
that schools don't look enough at v^ether teachers really like 
students." A willingness to give students a certain amount of freedom 
and responsibility, showing respect for students' opinions and ideas, 
and treating all students fairly and equally were other features of 
teachers' attitudes towards students that were iirportant for three or 
four students in each case. 

Instructional practices that commonly distinguished teachers v*o 
pushed students to their best effort were making the subject 
interesting, encouraging student participation, and challenging student 
thinking. While in the case of a particularly engaging class the first 
of these three was described by more than three tijnes as many students 
as the other two, in this context it was mentioned less frequently with 
all three being about equally ccr-imon. Exaitples of the first two 
practices already have been proviaed, \Aule exaisples of challenging 
students to think were Socratic quest^'-iing (e.g., "they keep on asking 
you until you get the question - tixay really make you think"), and 
"creative" assignments that "really make you think about wtiat you are 
doing." 

Teachers' academic es^sectations featured more prominently in 
students' discussions here than in their descriptions of the engaging 
class. In addition to teachers v4io made students do a lot of work, 10 
students in tota]. characterized teachers viho push them as either 
insisting that wpr? be done (and usually on time) or conveying a 
similar ejqpectation or desire without ±rposing- -stric± demands throu^ 
threats of punishment or other external means. S'crict teachers, for 
exanple, "check the [name]work everyday", "get on your backs to meet 
deadlines", and are "pushing, pushing, pushing, ens,uring you're working 
the vdiole time, watching over you in class." Other teachers, however, 
were reported to be more effective in eliciting other students' best 
effort by "sort of pressuring you . . . [by] tellijrig the consequences of 
vHnat will Jiappen, but not so much that you're stressed"; by "pushing 
you, but in a vay that makes you want to do it"; or merely by 
indicating in some v/ay that "they want you to work, but don't do 
anything specific to get you to work." Ttds distinction in the 
responses of different students suagssts that seme students need 
external pressure to do their best, while others are motivated by more 
subtle means, such as the student who said of her teacher "if you don't 
do the work it feels like you're letting down a friend." 
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Ihe three personal characteristics of a nice or likeable 
ijersonality, a sanse of humor, and enthusiasm for tcachit^ again were 
the most cctnmon responses in this category. Almost one quarter of the 
students mentioned likeability which represented the second hirfiest 
frequency of any category. In contrast, teachers' knowledge about their 
subject rated a mention by only three. 

■me low frequency of responses describing teachers' taiowledge basse 
as an outstanding characteristic (in classes which are engaging or 
rhose vdiere students put forth their best effort) contrasts with the 
enpbasis placed on this characteristic by the teaching effectiveness 
literature. It is possible, however, that the students in this study 
regard most of -iheir teachers as knowledgeable about their subject and 
therefore do not find it a distinguishij^ characteristic. 

Types of Lessen R mnats that Stisdents Find Enaaaire y 

Besides the qpen-ended questions on particular classes, students 
also were asked to indicate v^t types of lesson formats (in all 
' general, found interesting aivJ worthwhile. Tm out 

students agreed that student-centered class discussions (such 
JS>d "^^iS^f.P^^r^^ ^ engaging (see Table 6L). And another 

three of t^s ei^t listed types were checked by a majority: working on 
a ta^ jji a small groi^, class discussion led by teacher, and working 
on a task witii a friend. These four share a common instructional forraT 
namely one in vdiich students are active participants eraaged ii 
interacting with their peers (as well as in one case their teadier) . 

Class discussions in general (either teacher or student led) were 
©?>lained to be engaging for a number of different reasons. Some 
stuaents siirply enphasized that they are "a good way to get everyone 
involved." But besides enabling active participation in class, others 
stated you are "ejq»sed to different points of view." While this 
a^osure f or sane may satisfy just an interest or curiosity in hearing 
different opinions, other students seemed to value the oroortunitS 
discussions provided for social comparison, that is to see how their 
own views ocjipare with those of their peers: 

"You learn what the majority thinks. You hear other people's 
reactions to your points: whether or not they agjr^e. ... l feel 
bettea: when people have the same point of view as I uo." 
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•Bable 6L: Types of lesscais Students Export as Engaging 

(n=43) 



Rank 



Lesson Format 



Frecaiencv Percentage 



8 
9 
10 



1 
2 
3 
4 
5 
6 
7 



class discussion led by students 
TOrking on a task in a small groi^) 
class discussion led by teacher 
working on a task with a friend 
film 

depends on the class or teacher 
teacher questioning class on 
imderstanding of subject matter 
working on a task by inyself 
lecture 
other 



15 
11 
10 
1 



29 
24 
23 
22 
18 
16 



34.9 
25.6 
23.3 
2.3 



67.4 
55.8 
53.5 
51.2 
41.9 
37.2 



When disagreements arose, a few students, in contrast, enjoyed the 
ccaitpetitive diallenge of trying to win an argument wjth their peers: 
for exaitple, "if you r^ly 'zing' someone it bu Js your self- 
esteem." The challenge for seme, hohrever, was to ma.^ "yea think about 
your own" [view] and "understand [it] - which you must do to ejqjlain it 
to others." 

Students who preferred student-led discussions tended to stress 
that everyone was more willing "to say vtot's on their minds" because, 
according to one student, "you don't feel inferior as you do with a 
teacher," or because, according to another, "sometimes you are more 
willing to listen -to your classmates" (than the teacher) . As a result 
student-centered discussions were felt to produce more lively 
arguments. IVro students also mentioned the benefits of debates where 
you may have "to be on a side you don't agree with and then you're 
well-inforroed about both sides of the topic." 

Ihe advantages of teacher-led discussions, on the other hand, were 
described as having the teacher add "correct inforiration" or "facts and 
ideas students might not think of." Furthermore, tjie teacher "can lead 
it in interesting directions" and keep the discussion "orderly", 
focused and relevant which "avoids going off into things that don't 
matter to the lesson." 

Working on tasks in a small grcns£> or with a friend was perceived 
as engaging for similar reasons. Ihey provided "rrare relaxed" and 
"personal" environments in which it was "easier to express yourself" 
and "hear everyone's ideas" and "different opinions." Various students 
also claiii ' that in snail groups you get "more ideas" and you "come vsp 
with the besit answer by working together," and "if you don't understand 
you're more willing to admit you don't understand and can learn frou 
the others." Seme students seemed to find such conditions only came 
frc?^ working with a friend, vMle others only specifically described 
small groi^ in this \ray. Although many students said that it was easy 
to work with a friend because, for exanple, "you knew how a friend 
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thinks," scsne admitted that "you can easily get distracted because you 
want to talk about other things." 

One student made an interesting distinction between different 
subject areas in regard to the respective advantages of working wj.th a 
ftiend ani working alone. He r^rted ccmLng vp with the best ansv^rs 
in math and science by working alone because "you know ^tot you did" in 
analyzing and trying to solve a problem, ^diereas in social studies and 
English he finds he does better by "taJJdng back and forth" with a 
firiend. But another student stated that working on a social studies or 
English task alone enables you to "do vdiat you want to do." 

Films typically were portrayed as "more relaxing and easier to 
understand" because they "usually ejqplain \*at's going on more 
clearly, " as well as just providing a change of pace. Iteacher 
questioning ^^as enfiiasized (by approximately one third) as helping "you 
understand scHtBthing" (viiidh "makes it more interesting") getting 
"feedback from the teacher and the class," and makirg "you more 
ccsnfortable about givir^ your own qpinion" (after hearing the teacher's 
opinion or questions). Less than a quarter of the students r^x^rted 
lectures to be engaging, but those vAio did said they were worthwhile 
because, for exaitple, "you have to pay attention" and "you can get a 
better tmderstanding of vSiat the teacher is talking about if they sit 
vp and e}?)lain it to you step fcy step." Often, however, interest in 
lectures d^)ended on the particular teachers, scane of whcan "are 
boring." And apparently "some teachers can even moke discussions 
boring," but ""mostly the topics are boring." Engagement in class 
discussiOTS also for one student depended "a lot on the teacher becaxose 
scane teachers go too fast or esqpect you to know too much, \Mle others 
ignore half the class." 

Types of Writing Tasks Pound Enc^aq iirf 

Students also were requested to designate the types of writing and 
the subjects of writing assignments that they generally found 
interesting and worthxMle. In the formsr case, writing paragraphs or a 
short essay, taking notes, and writing a paper were each checked by 
approximately half the students (see Table 6M) . Almost as many, 
however, indicated that it depended on their interest in the topic or 
subject vdaether or not any type of writing was interestirg. 
Interestingly, the less substantial writing tasks of making an outline 
and writing one or two sentences were checked as engaging by the fewest 
numbers of students* 

Writing paragraphs or a short essay was reported to be interesting 
because "it gives you a chance to be creative and come vp with your own 
point of view," as well as worttoMle because "you have to select the 
most important points and write clearly and concisely." Many students 
seamed to feel tliat short essays are a good length because they don't 
require an excessive amount of information, but allow sufficient scope 
for "spreading facts around and giving different opinions" in a concise 
and specific form. Other students, however, found that short essays 
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enable you to "only get out one or two ideas, [vdiich] you can't really 
e>?>and," vfliereas in a pap^ "I can get all ny thou^ts out," One 
student en^jhasized ^ writing a paper "puts you in diarge of vdiat you 
want to do and leax*, adding that it is "a very personalized kini of 
task thav you are responsible for." 

Table 6M: T^^^s of Writing Students E^xsrt as Engaging 

(r^3) 

Rank Type of ^^itina Frequency Percentage 



1 writing paragraphs or a short essay 22 51,2 

2 taking notes (from a lecture or bock) 20 46,5 

3 writirg a paper 19 44.2 

4 depends on the topic or subject 18 41.9 

5 making an outline 13 30.2 

6 writing one or two sentence answers 4 9.3 

7 other 3 7.0 

8 none 1 2.2 



Taking rK>tes was stressed generally as beirg worthwhile rather 
than interesting. The reasons cited were tiie necessity for 
"concentrating" or having "to read the material carefully and 
understa2id it before you can re-^^rd it," or the advantage afterwarxJs 
of having the notes to refer back to (especially helpful in preparing 
for tests, or to enable you "to see the progression of ideas in a 
class"). One student tried to e)5)lain how both the process of taking 
notes and the end product helped you learn: 

":mere is something about taking notes that helps you learn, I 
can't^ ejqplain it. It helps you because you^re taking it in and 
learning scanething new. Having background notes on topics you don't 
knew anything about malces learning so much easier." 

Making an outline was regarded as worthvMle for sonsaAiat similar 
reasons as those given for taking notes. In other words, these students 
reported that actually making an outline helped them understand and 
organize the main points or ideas, while the conpieted outline "helps 
when taking a test and writing an essay •" Typical also of the comments 
of several students was the claim that it is "easier than writing the 
paper itself, you deal with major ideas only ard not specifics vMch 
take more effort." 

In response to the list of general subjects of writing 
assignments, taking a position on an issue and defending it was the 
most popular, being checked by approximately two out of three students 
(see Table 6N). Describing personal ej^^erienoes or feelings and 
explaining a concept, principle, theory or problem were each indicated 
as engaging by about half the respondents, with creative writing 
popular with just less than half. 
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OSable 6N: Subrjact of Writit^ Assigniaents Students Eepcoi: as Engagiiig 

Rank T ype of Writincr Assignment Frequency Perx:gntaae 

1 taking a position on an issue ard 
defending it 

2 descaribing in/ e^^^eriences or feelings 

3 e}5)laining a concept, principle, 
theory, problem or issue 

4 creative writing (making up stories 
or poems) 

5 reporting on independent research 

6 summarizing material covered in class 
or readings 

7 depends on the topic or subject 

8 other 

9 none 

Ihe challenge involved in aiguing for and defending a position on 
an issue seemed to underlie most students' explanation of why this type 
of writing assignment was engaging. For example, students commented 
that "you have to take into account arguments of the exposition and 
counter them with your arguments," "it's your own point of vie^^r and you 
try to persuade the reader to come to your point," and "it's 
challenging and you have to stay on one side of the argument the ;*ole 
tiioe." And "if yoa can't give reasons vAiy you feel strongly about your 
position, then you realize you don't think as strongly as you believed 
you did." The challenge was not limited for some to defendii^ their own 
position, because "even if you don't believe in it [the position], you 
can put forth the effort and discover you can argue for soffivething you 
don't believe in." 



28 


65.1 


22 


51.2 


21 


48.9 


19 


44.2 


13 


30.2 


11 


25.6 


9 


20.9 


2 


4.7 


1 


2.3 



Both journal and creative writing were appealing to many students 
because of the freedom they provided. Students said they didn't hwe to 
worry about: "spelling and grammar," "using notes or an outline," 
"follcwii^ an assigned topic," "sticking to facts", and "definite 
answers that the teacher is looking for." Instead, th^ could "just 
make things \jp/^ "write how you talk," and write "vtotever comes into 
your head." Describing personal experiences or feelings enabled 
several students to express and reveal pa2± of themselves, and one 
student to "get a load off your back and . . . think about and deal with 
iry personal problems." Another student provided a compelling account of 
hew journal writing enhanced the teacher^student relationships in a 
dass because of the teacher's interest in "our thou^ts and feelings." 
Furthermore, she added that discussions were better because "people in 
the class opened vqp more," for example, "if talking about a novel, 
they'd be more likely to share their true feelings." 

Ej^laining a concept, principle, theory, etc. in writing was 
reported to be worthwhile because it helped you \anderstand the 
material. Olie one quarter of respondents vftio checked summarizing 
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materi.al generally indicated that it was worthvMle for the sane 
reason: for exanple, it "helps you understand what someone is saying if 
ycai have to summarize it." lliose vdio found reporting on independent 
researdh interesting enjoyed the qjiportunity to choos?. their own topic 
vMch often cjxorred with this type of assignment. The research for 
this writing was regarded some students as worthvMle because "you 
learn a lot about the subject" and "you're sharpening your research 
skills vdiich are important irrespective of vftiich field yai enter." 

Social studies was the most frequently mentioned subject vdien the 
students interviewed in this stud/ were asked about the most engaging 
course they had taJcen in the current school year and abcut a 
particularly engaging class they had e>5>erienced in hi^ school. 
Science, English and mathematics were the only other subjects mentioned 
by more than a ca:ple of students. 

In prcbing students' reasons for being engaged by these classes, 
and by lessons or tasks in a social studies course dbseirved by a 
researcher, the subject matter and/or the activities were mentioned by 
the vast majority in each case. Half of them described the subject 
matter as intrinsically interesting (especially in the case of social 
studies and often because of its perceived relevance to the real 
world) , and half reported that the opportunity to actively participate 
in class (mainly throu^ discussions in social studies and English, and 
throu^ manipulating information or things in science, mathematics and 
other subjects) made the course or lesson engaging. In the case of the 
social studies course, many cited as engaging lessons Where they were 
learning scanething new. Many students also attributed their engagement 
specifically to the teacher, particularly his or her ability to make 
the class interesting or challenging. 

When asked to describe some of the most interesting activities in 
a particularly engaging course and the nature of a particularly 
engagjjTg lesson or assignment in their social studies course, a writing 
task was the most popular type in both contexts (closely followed by 
discussions in social studies), ihs content involved in most students' 
exaitples concerned main ideas and either values (in the social studies 
course) or pecple's ej^jeriences (in the case of an engaging course), 
rather than discrete facts. Somevtot surpr-'singly many of the engaging 
exaitples were homework assignments. In tjth cases, but especially in 
tlie social studies course, most students outlined activities requiring 
them to manipulate or irake sense of information, vMle a nimiber 
described activities concerned with acquiring information. However, 
v^en questioned about specific social studies lessons observed by a 
researcher half the students mentioned as engaging questions or tasks 
mvolying the acquisition of information, with just under a half 
describing questions or tasks requiring cognitive manipulation (mainly 
making inferences or value judgements) . 
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All but one of the students vfao had experienced in--d^)th work 
(i.e., stiKJying a specific topic, question or prdDlem for two or more 
weete) fcwnd it interestirig. Generally th^ felt it was interestiiig 
because they liked either the particular tqpic or the opportunity to be 
creative or to work on their own. Many also est^iiasized hew much they 
learned frcsn doing in-d^±h work. However, they were almost evenly 
divided on the question of viiether they would like to have jsare in- 
dqpth wor^v school. 

When asked about any outstanding characteristics of the teacher in 
a course they found particularly engaging, about a third of the 
students described the teaciher * s ability to make the siibj ect 
interesting (for exanple,^ relating interesting stories or anecdotes, 
or assigning fun activities) and a third mentioned various other 
instructional practices. Attitudes towanJs students (such as being 
concerned about helping students leam and giving them a certain degree 
of freedom and responsibility) and personal characteristics (such as a 
sense of humor ai^ a likeable disposition) were each listed by nearly 
half the students. Instructional practices (including encouraging 
student participation) were again mentioned by most in relation to 
teachers who push students to their best effort, with the same nuniber 
discussing attitudes tcwards students, especially a teacher's corcem 
for helping them learn. While academic es^jectations were rarely 
mentioned in the case of an engaging course, this time many students 
made scane reference to teachers' academic demands. 

In terms of general i^pes of lesson format that are interesting 
and worttoMle, class discussions led by students were the most 
popular, followed by small groiqo work, class discussions led by the 
teacher and working with a friend. Class discussions seeaed to be 
ei^ging for a number of different reasons, ranging from creating 
student participation to enabling social ccstparison with peers. Wbrking 
in a small groip or with a friend v;as regarded as personal and 
relaxing. 

As far as writing tasks are concerned, writing a short essay, 
taking notes and writing a paper were each indicated as interesting and 
worthvMle by approximately half the students. The lei^th of short 
essays was popular with scsne students, vMle others preferred the 
greater scope available in a paper for expartiix^ on their ideas. ^Jaking 
notes was strictly seen as worth^^Mle for tttilitarian reasons, namely 
facilitating coiKrentration on the material and ser/ing as a reference 
vdien stud/ing for tests. Two out of three students indicated that 
taking a position on an issue and defending it was an engaging kind of 
writing assignment because of the challenge involved. About fifty 
percent in each case thought journal, creative and ejqjository writing 
(about a concept, principle or theory) was engaging. Journal and 
creative writing were appreciated for the freedom they provided from 
wrrying about having the right information and answers, while 
ej^xjsitory writing was desribed as valuable for helping ycu to 
understand the material. 
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Many previous studies have indicated that hi^ school students 
generally find classes boring and put iniiiirnal effort into their 
academic activities (Csikszentmihalti & Larson, 1984; Cusick, 1973; 
Goodlad, 1984; Sedlak et al, 1986). One of these studies 
(Csi]cs2entinihalti & Larson, 1984) , however, had found that stridents, on 
average, had at least one class \Mch was engaging. Similarli', in this 
stui^ the students were able to discuss at least one class'> and one 
academic task vd:ere time had passed quickly and they had made their 
best effort. Furthermore, their engaging academic e55)eriences had 
siinilar attributes; that is, most students tended to be engaged by 
similar kinds of ^ academic work arsd teacher behaviors. Students 
indicated that activities such as discussions and writing tasks that 
require active mental effort and teachers jAio make the subject 
interesting and convey a caring concern for their learning are likely 
to engage ttiam. In particular, insti actional formats jAiidtx involve 
students in interacting with their peers (e.g., debates, role plays and 
small group work) or the teacher (e.g., class discussions), and writing 
tasks which require students to take and defeixi a petition on an issue 
are appealing to the najority of students. 

Beyond the descriptive characteristics of engaging academic work, 
the more fundamental question is: What are the underlying causes of 
student er^gement? Despite popular conceptions , extrinsic rewards 
(sucii as hi^ grades, college admission or future job prospects) were 
reported by only a few students as a factor in creatirg engagement. 
Instead, the intrinsic value of learning was cited consistently across 
different academic contexts by a substantial majority, students valued 
academic learning because the content was interesting, the process was 
actively involving, or the outcomes were worthvMle. Ihe content was 
usually interestijjg because it was relevant to concrete hman events or 
to a student's idiosyncratic interests and concerns outside school. 
Oiatcones were worthwhile if the student learned scmething new or 
developed a better understanding of the material being studied. 

Different students were intellectually cihallenged by different 
kinds of school work. Essentially the students in this study were 
equally divided between those vflio found hunanities subjects (i.e, 
social studies arri English) challenging and those vto were more 
challenged by science and mathematics. In mar^ cases it was the 
different types of problCT^ associated with each of these subject areas 
that distii^uished students » perception of challenge. Die former grot?) 
tended to r^rt that ill-defined problems with problematic solutions 
made them think hard, vfoile the latter groop often found well--defined 
problems with single, correct answers more mentally demanding. Another 
distinction was between the demands for inductive inferencir^ in 
humanities subjects cited by some students and the demands for 
understanding abstract concepts in science and mathenatics mentioned by 
others. In short, students perceive the major challenges in English and 
especially social studies to involve the making of value judgements and 
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(usually inciuctive) inferences; and the major challenges in science and 
math to be making deductions and abstractions. 

As far as specific activities and assignments are concerned, 
teadier-directed activites that required student responses (e.g., 
teacher-led class discussions and teacher questioning students on 
understanding of subject natter) and suhetantive vnriting tasks that 
were ocnpleted individually cwtside cleiss were most ccsnroonly described 
as mentally demanding, especially when they dealt with ideas and/or 
values. In the latter case, the noteable exairple was papers or essays, 
vMch for sane students were challenging \*ien they involved ej$x3sitory 
writing and for others v^en ejqpressive writing was required. 

Again the important question to consider is: What makes these 
kinds of academic work oognitively challerging? Demands for hi^er 
order cognitive processing featured most prcradnently vAien students were 
asked if they ej^jerienced any frustrations. But these frustrations were 
apparently overccsne since the most common satisfactions were from 
successfully acccatplishing the task and from developing new insists or 
better understandings. This suggests that a challenge must be of 
sufficient difficulty to create an initial frustration, but not so 
difficult that it cannot be met - presumably with reasonable effort. 

Althcu^ teachers have claimed that secondary students passively 
resist intellectually challenging tasks (Hanpel, 1986; McNeil, 1986; 
Tye, 1985), little evidence was found to si^jport this contention. On 
the contrary, the kinds of academic work most students r^rted as 
mentally challenginc' were also generally the kinds they found engagirg. 
Essay writing tasks and teacher-led discussions dealing with ideas and 
values were frequen^iLy cited as both engaging and challenging (althou^ 
student-led discussions and working with peers were engaging but not 
challenging). Additionally, many students are both engaged and 
challenged by classes that eirphasize hi^er order thinking. 
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lEEaRIMENrAL AND ERIRC33>AL lEAEEESHIP IN H?OMOTTNG 
HIGHER CSSSR THINKENS 

Gsaneroi MbC3arthy 
Erancds Scfarag 

Many factors beyond the classroom may affect the cnoality of 
thinking in social stvidies classrooms. Do the principal and department 
head have a pronounced influence on \*iat happens at the classroom 
level? E>ctensive questionnaires and interviews with admimstrators (as 
well as teachers and students) in the schools we studied, give xas an 
opportunity to address this question,^ Here we report scsne of the 
results of our investigation into the role of the department heads and 
principals in the five schools. We are still in the midst of 
coTppleting our analysis of tlie iirpact of these actors on teacher's 
perceptions of si:?)port for hi^er order thinking and on actually 
occurs in tlie classroom* At this point, however, we shall concentrate 
on profiles^ of the leaders at the department and school levels and 
identification of some similarities and differences between them. 
Since we are able give a rough indication of the relative success of 
the five schools in promoting hi<^er order thinking in social studies 
classes, we conclude with some speculations regarding the iitpact of the 
principal and department head on the classrocms. 

I. 

Department Chairs 

A convenient place to start is with the department head who, in 
most hi^ schools, plays an inportant role in hiring teachers, in 
assignment and development of courses, in selection of texts and 
examinations, and in setting the tone for the social studies program 
and department. We shall try to identiQr the nature and extent of 
si5?port that each of our five department heads gave to their teachers 
in prcmoting thou(^tfUlness. 

Ihe five schools are very different in a number of important 
respects—geographical location, size of departments, number of periods 
in the day, average class size, and so on— but perhaps the most 
striking thing about the five social studies departnvent chairs is the 
extent to vAoich they agree on the principal satisfactions and 
limitations of their role. Almost all find their greatest satisfaction 
in watching successful teachers who have profited from their advice and 
siKjport. On the other hand, almost all find dealing with recalcitrant 



Respondents may strive to put themselves and their 
institutions in a favorable light. But our assurances that the 
names of people and institutions would be disguised in the final 
report gave those we interviewed the opportunity to speak with 
considerable candor. 
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teacOiers vdao are not doing a satisfactory job to be their major sour^ 
of fnistration. The administrative side of the job — responsibility for 
meetiiigs; paperwork, budgets, etc. is viewed by nost as a necessary 
evil; only one irientions it as a source of gratification. 

D^>artin9nt heads are also veteran teachers, and see themselves as 
peers. Althou<^ aditdnistratively superior, they have little actual 
pcwer over their colleagues. Hence, their primary source of influence 
lies in their ability to win colleagues over to their side, to persuade 
and cajole rather than conmand them to go along with a particular 
program or point of view. It is no wonder that all five inention skill 
in interpersonal relations as the principal requisite for a successful 
departitvent head. Neither is it surprising that they see themselves as 
most effective in informal, d/adic or small-grrr:?) interactions rather 
than in staff meetings. 

Despite ccanmonalities v^ich result primarily from the nature of the 
departuvent chair's role and working environment, differences among the 
five department chairs we interviewed can be readily discerned. We 
shall briefly describe each of them. 

Bob Mikkelson's title is "district coordinator" for social studies 
in Grandville, a wealthy, suburban school district in the Northeast 
region of the country. He has been in this position for 10 years and 
has been teachir^ social studies for 18 years. IMer his leadership, 
the district's hi^ school with 2000 students, (irrieed the entire K-12 
system) has been involved for the last 3-4 years in the development of 
a program to promote hi^er order thinking in the social studies area. 
Oliis is by far the most systematic and continuous effort amorg the five 
schools in this area. Mikkelson is the only departnvsnt head that one 
could really call the architect of a program. 

Progran development included the following activities: 

1. SoJ.iciting the assistance of a number of consultants, 
of whcan the most influential has been Barry Beyer, v^ose 
ejqplicit "skills" jrodel of thinking has been adopted by 
the department. 

2. Visiting other schools in the district and conductir^ 
a needs assessment within the school. 

3. Designing an overall curriculum content and skills 
sequence for K-12, and revising a mandatory U.S. History 
course to incorporate the "thinking skills" approach. 
Formulating lesson plans and teaching demonstration 
lessons to staff. 

4. Organizing systematic peer observation throu^iout the 
departit^t. 

5. Ordering materials and designing criterion- 
referenced tests to reflect the new focus. 

Ihou^ much of this work was done by committees under Mikkelson's 
leadership during the regular school year, it is ijrportant to note that 
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the work cxantinued during the sumnveL months. Teachers were paid $75,00 
a day for participatii^ in developing the program; Mikkelson calls this 
the '»inost valuable" money. 

like all d^)artment heads, Mikkelson is responsible for observing 
teachers. He is required to make at least three visits to each of 40 
district social toadies teachers in a given year. To be able to carry 
on a vast jprograra of curriculum development, vMle performing his other 
siqpervisory duties and teaching a class of his cwn, clearly requires 
vmusual energy and dedication. Mikkelson believes that all his staff 
share his vision of excellence, thoug^i a few are not sure they have the 
tools to iirplement it. Mikkelson }y±es that he has helped shape and 
lead to fruition a major program change, and that he can go into 
classrocsns and "see it beirg done." Hds he feels is the itost 
rewarding part of his job. 

Sam Holmberg is a veteran of 34 years teaching social studies. For 
tb;£ last 17 of these, he has been department chair at Carlsberg Kic^ 
School, a school of almost 3000 students in a major city in the 
Northeast. While Sam Holmberg is also enormously skilled and 
energetic, his approach differs from Bob Mikkelson in two significant 
respects. (1) Althou^ he and his teachers have bcien involved in the 
revision of particular courses within the program, tliere is no e5q)licit 
enphasis on thirikir^^ p^ Carlsberg doesn't clistinguish between 

instruction in thinking and good social studies instniction ^Adch 
"calls for thou^t-provoking questions and hew and vAiy questions." (2) 
"While Mikkelson *s eitfiiasis is on the broad outlines ctnd articulation of 
the elements of the entire social-studies curriculmi, Holmberg 's focus 
is on the individual lesson. If Bob Mikkelson is an architect , Sam 
Holmbejcg is a mentor . 

Ihe itrprovement of lessons is worked at fixm several directions. 
HoLtibarg does a great deal of observation, visiting each teacher at 
least twice a year, and non-tenured teachers at least, six times a year. 
He writes detailed reports and conducts follow-i?) conferences with 
teachers conceming dbseirved strengths and weaknesses. He, other 
teachers, even the principal on occasion, teach "demonstration lesson5=i" 
to regular classes with colleagues in attendance. After the lesson, 
there is a critique session in vAiich strengths a:ad weaknesses are 
discussed. But the designing of the lesson plan receives greatest 
attention. Each lesson ph.w is a product of groi?) effort, and every 
lesson plan tau^t is available to teachers in a resource file. 
CouL'ses are perceived as sets of self-contained "les'^sons", so a major 
route to course iirprovement is via inproved lesson plans. 

Sam Holitiberg, like Bob Mikkelson but in a less formal way, creates 
a climate in vMch shariiig and collaboration are the norm. Holitiberg 
sees his role as disseminator of critical ideas and teachincn 
techniques. Asked about the most rewarding part of his job, he says', 
'fWatchii^ teachers perfect thrir teaching techniques is ty far most 
important. It has been a tremendous pleasure to see Matt O'Brien 
beoofftve the great teacher he is. Olie specific pleasure coenes from the 
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exchange of ideas and the planning that is an iirportant part of 
teaching." 

Herb Dexter, another veteran teacher of unusual energy, is 
department chair at Arnold High, a 1100 student hi^ school in a 
* middle-class subui±) :>f a major West coast city. While teaching a full 
load of courses (five per semester in a six period day), Dexter has 
pioneered the development of an eclectic approach to teaching thinking 
vMch he calls the "Integrative Mind Instructional Model." A striking 
feature of the model, develqped collaboratively with one of his 
teachers, is the en?Dhasis on metaphors and analogies vMch students 
create and represent visually in different colors. Like MiWoelson at 
Grandville, Dexter has read the conceptual literature on thiriking, and 
has himself given workshops at other schools on his own approach to 
promoting thinking. He's also been involved at the state level in the 
creation of a new American History exam. Because the exam will 
enjtosize thinking through issues. Dexter does not perceive it as a 
threat to his own enphasis in social studies. 

Dexter sees himself and tries to act as a catalyst for others, to 
prod, stimulate, and si^jport their efforts to excel as teachers. 
Because he teaches a full load (He was teaching 174 students each day 
in five classes vto we interviewed him) he has neither the time nor 
the^ mandate to formally observe other teachers. Considering the 
limited free periods a day (each teacher has one) , the enormous student 
load each teacher carries, the lack of formal supervision or peer 
observation, and the paucity of si^jplies. Dexter 's ability to 
disserdnate his vision and his ideas is remarkable. He claims that 85- 
90% of the teachers now share his general orientation. 

Herb Dexter tries to share ideas with teachers in informal 
interactions, in curricula planning groii^js which he organizes, and at 
formal department meetings which l^ically focus on pedagogical issues. 
He encourages others to take risks in their own teaching by revealing 
his own attenpts to innovate, even vdien they're not successful. He 
treats everyone, even student teachers, as peers from vAiom he can learn 
something, ihe most iirportant quality a dqpartn^t chair needs is "the 
intangible of inspiring people to keep shooting for the moon— trying to 
improve." Speaking of himself, Dexter says, "I am a si:?5porter of 
others' rise to excellence—the role I cherish and sometimes abhor v*ien 
my own intellectual commitment to a professional identity as a 
historian or political scientist emerges." 

Ixmse Ellsworth, an especially thou^tful teacher, is the only 
woman in this group. Bradley Hi^ Sdiool, where she is a chair of 
social studies, is a 2100 student school in an older suburb of a large 
city on the Eastern seaboard. Its students display a diversity of 
bad^round and ability not found in most suburbs. Ellsworth has been 
chair only three years. Prior to that, there has been a history of 
turnover in the role, due, she believes, to a tradition of individual 
faculty autoncaity at the school yhidh undermines the chair's ability to 
set and carry out policy at a departmental level. Ellsworth has a 



ERLC 



^205 



VII - 5 



general vision of vfliat an ideal program to develop thou^tfulness voxLd 

look like, but she is not naive enou^ to believe that inost of the 

other teachers are looking for someone to lead them. One of 

Ellsworth's hi^est priorities is siirply to develop a more 

collaborative culture in the departrant. A new rojuired 

course for sophomores will provide an opportunity for teachers to work 

together. 

Ihe school system does not encourage systematic program development 
or in-service activities to inprove pedagogy. In 1984, the department 
had some workshops with leaders of the "thinking skills" movement, but 
Ellsworth says only a few teachers really showed an interest. To the 
extent that the teachers do err?3hasize thinking in their classes, this 
results frcan their own personal orientation rather than from any 
department-wide efforts to diange curriculum or pedagogy. 

Ellsworth is required to confer with both non-tenured and tenured 
teachers several times a year, and these sessions provide the main 
official vehicle for working with teachers. Ellsworth bfdieves that 
discussions about curriculum, approaches with students, or evaluation 
can be even more val\:iable than cibserving classes. "Hew much can you 
really tell a teacher about new to iirprave little moves in ttie 
classroom? Since irost of the teacher's work is done before or after 
class, this is \*at demands more attention." 

UWLike most of the other department chairs, Ellsworth has had the 
opportunity to hire a significant number of new teachers recently— 
ei^t. With young teachers, she feels her main task is to be 
siqpportive, to convince them that "every lesson doesn't have to be 
perfect." With older teachers vdio are comfortable with one style of 
P^gogy/ her main challenge is to help them remain open to alternative 
ajproaches. "I encouraged a fairly rigid teacher vAio loves the 
Socratic method to use journals for kids to reflcx± on their own 
thinking." 

We choose the label therapist to characterize Louise Ellsworth 
since inuGh of her success with her staff as well as her orientation to 
teaching derives from her intuitive understanding of sound therapeutic 
principles: the need to build trust so that others can open vtp and 
share their shortcomings, the need to overocme resistances to new 
approaches and perspectives, the realization that change results from 
people being able to take a fresh look at themselves, not fran 
supervisors pointing out their mistalces to them. Not surprisingly, she 
feels the least rewarding part of her job is 'Working with teachers who 
are afraid to look at themselves." 

Mike Ereminger, the final member of our groip, is social studies 
chair at Scarborou^ Hi^, an 1000 student inner-city hi^ school 
located in a major Eastern metropolis, a magnet school vMch plays a 
key role in a system-^ide reform effort. A general focus on developing 
critical thinking for all students has been introduced at the district 
level, but it is institutionalized primarily throu^ a set of writing 
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exercises ard short discussions that are not systematically integrated 
into the curriculum of any departitvent. Preminger was one of ten 
teachers fran the district vSio served on a planning ccanmittee in 
develcpnent of a US History course. 

In the social studies department, all three mandatory year-long 
CG?'>rses are required to follow unit packets designed by a team of two 
university professors and twenty teachers. Qiis can substantially 
xmdercut the aiitoncaity of the individual teachers. 

Almost all the social studies teachers, are veterans and were 
specially selected for this flagship school. Despite the attenpt to 
develop a district-wide, "syllabus-driven" curriculum, the teachers 
value their autonomy and, like those at Bradley, do not eq^^pear to be 
looking for guidance or leadership. "Ihe lone-ranger syndrome can hurt 
us," Preminger says. Althou^ required to observe and confer with his 
teachers, Premir^er feels this activity itay not have much inpact. 
Althou^ he relished the chance to observe and give xiseful, positive 
feedback to a ftiex>d and colleague, Pi-eminger feels he can best assist 
his staff by scheduling classes according to the strengths and 
interests of the teachers, by securing materials for xise in their 
classrocjns and by clearing other bureaucratic hurdles. 

Unlike Bradley and Arnold, i*iere there is little pressure frcan 
above, here at Scarborough there are constant demands and e>5)ectations 
fec3m the principal and district. In the interviews, Premiriger 
constantly referred to vAiat "they" want. His main frustration is that 
althou^ he was picked, in part, to develop a new interdisciplinary 
curriculum in international studies, the effort has never received the 
leadership frcm the school administration needed to bring it to 
fruition. Yet, of the five department heads, Preminger is the only one 
vdio claims that administrative activity is satisfying. "Ihe materials 
ard budgeting are very iirportant. it gives me a better sense of 
control — especially the chance to find interesting materials that I can 
pass on to colleagues." His role appears to be primarily that of 
intermediary relaying mandates from above, and requests from below. 
His leadership in curriculum development or promoting new approaches to 
teaching appears very limited, but it is hard to determine vfiiether this 
is due primarily to his non-directive approach or to demands from above 
vihidi seem to afford little opportunity for axitonomous action at the 
departmental level. 

Having briefly described each of the chairs and d^)artments, we may 
observe that several have common features vMch not all share. In 
Qrandville, Arnold, and Carlsberg there is a real sense of 
collegiality, of individuals not only working tcwards coatimon goals, but 
also sharing ideas and helping each other perform better. Ihe focus is 
on instruction, and the department heads in each of these sciiools are 
clearly perceived by the teachers as master teachers from vfeom they can 
learn. In Bradley and Scarborou^, on the other hand, there is less 
sense of a common enterprise. Most of the veteran teachers are not 
looking for guidance from peers or si:periors. 
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Another pattern is revealing. At three of the schools, Arnold, 
Carlsberg, and Grandville, their social studies department had a 
systematic^ program aimed at promoting hi^i^ order thinking. In only 
one case, Grandville, had they adopted the "thinking skills" 
orientation. Sca3±)orofu^, thou^ it had school and district-wide 
programs focused on thinking, had no program that originated in the 
social studies department. Neither Scarborou^ nor Bradley manifested 
an explicit concern with hi^er order thinking at the d^artmsnt level. 

Finally, it is worth noting, that only at Grandville, was there a 
major ccaranitinent of resources, both tiine aid material, to both develop 
and iitplement a viiolesale curriculum reform. Inadequate time for 
planniryg, too heavy a teaching load, lack of supplies of resources, 
failing equipment, seem to be common in the other schools. 

n. 

Priiicipals 

As we found in the case of the departiaent heads, there are 
ccaranonalities in the principals* conceptualizations ard efforts in 
si:5^rt of hi^ier order thinking in the five schools we studied. But 
as we shall see, there are iitportant differences as well— particularly 
with re^)ect to administrative style and type of support provided for 
thinking. Ihese differences in styles and approaches also influence 
the quality of collaboration and cooperation between the principals and 
the department heads. Ihere is, for instance, a_hic^ degree of 
continuity and collaboration on agendas for instructional irrprovement 
and methods of evaluation between the principal and department head at 
Carl^^erg. On the other hand, there is significant discontinuity 
between the perspectives and agendas of the principal and departn^t 
head at Scarborou^. 

But first, let us discuss some of the coiratonalities among the 
principals. Even thou^ there is little significant e>5)osure to recent 
research literature, there is a remarkably shared vision among the 
principals of what a hi^ier order thinking classroom should look like. 



^According' to cur definition, a systematic departmental-based 
program aimed toward hi^ier order thinking should reflect three 
criteria; members of the department should share a coamon 
conception and vision of higher order thinking; lesson plans, 
syllabi and other materials shaold relate the conception or vision 
to the lessons tau^t; and there should be interaction among 
department members on a regular basis that is focused on how well 
they are progressing toward the vision. Departitvsntal programs can 
conceivably be grounded in district level or school-wide programs, 
but tmless we found indicators such as these at the department 
level, we did not recognize the existence of a si'stematic, 
departmental-based effort. 
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Ihis vision involves: less teacher dcaninated classrocans; teachers 
posing challenging questions or tasks; teachers allowing students to 
tackle questions from diverse perspectives. Four of the five 
principals r^rt that their respective social studies departments are 
ahead of other subject d^)artments in terras of classroom instmction in 
thirikii^. Two principals felt that a number of social studies teachers 
at their scshools were not good "thinkers" and consequently were not 
good locxiels of thou^tfulness for the students to emulate. But in 
neither of these cases, did the principals mentioned report direct 
involvement or intervention on their part to help iitprove the classroom 
instruction of the teachers identified as inadequate. Day-to-day 
instructicml iirprovement in these two schools is seen by the 
priiicipals as the responsibility of the department chairs. 

All five principals attach an enorroous significance to school 
"tone" ard the fostering of an academically disciplined environment as 
preconditions for thinking ("we need an orderly and safe school 
concomitant with academic achievement," Lester Brown, Carlsb^rg). It 
is, however, significant that vftien asked about their goals, none of 
the principals lists thinking as his first priority. *Ihe three most 
consistently mentioned goals ars: (1) iitproving school climate; (2) 
iiiproving/maintaining academic acihievement as reflected in test scores; 
and (3) teacher trainii>g. 

Barriers to thinking — Ihe principals also agree about \A]at are 
the main barriers and constraint:s on their efforts to promote thinking, 
like the d^>artment heads, the principals report that administrative 
demands ("paper i.'crk") have been increasing over the years. These 
demands limit the amount of tine they have to address curriculum and 
instructional issues in their schools. Three of the five principals 
(Arnold, Carlsberg, & Bradley) conplain of scarce resources and lack of 
funds for instructional iitprov»2ment and staff developnent programs in 
hitler order thinking. At all five schools, curricular materials for 
higher order thinking, particularly teacher produced materials are 
considered scarce or inadequate. The principals also reported pockets 
of apathy and resistance to initiatives to hi^ier order thinking among 
members of their staff. Ihou^ the principals readily express 
frustrations with scarcity of lesources and funds, and constraints on 
their time to address instructional iirprovement, principals rarely 
mentioned organizational and structural features of their schools as 
significant constraints on hi^r order thinking. For the most part, 
the principals seem to accept as given the routine features of school 
organization, viz. , class size., scheduling, and structure of the 
subj ect disciplines . Tliey do not eirphasize organizational or 
structural refonti as a strategy for achieving hi^er order thinking in 
their schools. 

Administration versus instructional leadership — Even more so 
than the departtt^t heads, the principals e55)erience a critical role 
dilemma — a dilemma that is inscribed in the designation of school 
priiKxipal. This dilemma is expressed in terms of a tension between the 
demands placed on them as administrators versus demands placed on them 
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to be assertive curricailvmi and instructional leaders. Paul Benton at 
Arnold puts the matter rather forthri^tly: "Administrative 
reorganization in the district has shifted ny tiiae away f rem curriculum 
issues to administrative work." Paul Benton's social studies 
deparbnent head. Herb Dexter, also mentions excessive administrative 
demands as a barrier to promoting hi^er order thinking instruction. 

But it VTCJuld be wrong to concl\2de from the above observations that 
the principals are not involved in instructional iirprovertent. All five 
principals ^ress a particularly strong interest in instructional 
issues and indicate that they make efforts to reconcile the dual 
obligations of administration and instruction. The priiKjipals all 
report that they use administrative cont2x>ls over budgets, resources, 
personi^ selection, agenda setting, and evalxiation to influence 
instructional objectives in social studies .and other subject 
departments at their schools. Ihree principals, Quentin Donate at 
Grandville, Ken Olds at Scarborou^, and Martin Broderick at Bradley 
indicate that they use primarily administrative leverage, particularly, 
over personnel selection, to secure curriculm and instructional 
c4>jectives. Ihe other two principals, Lester Brcwn at Carlsberg and 
Paul Benton at Arnold, in addition to carryirg out administrative 
duties, are directly involved in the fostering of instr\:ictional 
inprovement and staff developnent programs at their sdiools. Unlike 
their counterparts at Bradley, Scarborouc^ and Grandville, the 
principals of Carlsberg and Arnold see themselves, first and foremost, 
as instructional leaders. 

Differences in Styles and Aijproacfaes to the Fosteriirr of ^hirikiixf 

Leadership styles and approaches to promoting thinking vary 
significantly among the five principals. These styles and aj?)roaches 
to thinking are influenced by such factors as school history/tradition, 
district policies, depa2±ment head and principal relations, and the 
general climate within the school. We illustrate this variability by 
briefly describino :he context within vMch each of the five principals 
operates and the salient aspects of each principal's leadership style 
and method of siqpporting thinking in school. 

Lester Brown at Carlsberg works in a school environment that can be 
described as remarkably stable— this despite the fact that the school 
population is hi^y diverse~-50% vMte, 22% black, 21% Asian and 7% 
hispanic. As was allvided to earlier, there is a very low rate of 
turnover among the faculty at Carlsberg. Ihis is especially so in the 
social studies department vdiere the average number of years teaching at 
Carlsberg, among our sanple, is over 15 years. On the other hand, 
Carlsberg does not have the level of monetary and technical support for 
instructional inprovement from its school district as we will see in 
the case of Scarborough and Grandville. Indeed, Carlsberg 's social 
studio department head, Sam Holmberg, conplains about the inadequacy 
of funds and resources for the promotion of higher order thiriking 
instruction in his department. 
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What stands out abcfut Carlsberg, hcwever, is its strong sense of 
cxsnmitroent to student achievement and academic excellence, The sense 
of ccoranitinent is shared by both the staff and students. Ninety percent 
of the students at Carlsberg go on to college. Students excel in 
district and statewide exams and conpetitions for scholarships and 
awards. This eitphasis on academics is the Carlsberg "tradition," the 
principal suggests over and over again. He notes with pride, "we are 
an academic hi^ school. If you come to Carlsberg, you'll achieve and 
go to college. All students will get an academic core curriculum." 
Brown takes direct responsibility for instructional leadership at 
Carlsberg. 

Like his four counterpa2±s, the principal at Carlsberg enplqys 
general administrative strategies sudi as personnel selection and 
budget control to secure his curriculum and jjistructional agenda. 
Ktewever, Lester Brown's main strategy for prconotir^ hi^er order 
thinking is to be distir^uished in one pa3±icular respect: Carlsberg' s 
principal places enormous errphasis on thorcfu^ and meticulous 
evalmtion and instructicaial sipervision of his staff. In this 
respect, his collaboration and close working relationship with his 
department^ heads, particularly, his social studies department head, 
Hblitiberg, is crucial. Lester Brown holds workshc^ with his dqpartmsnt 
heads "to help iirprove and refine their methods of observation and 
assessment." He does over 100 cibservations (including elaborately 
written summaries/critiques) of his teaching staff every year— itore 
than any other principal in our saitple. In addition, he reads and 
ccfftiments vspon the lesson observation written up by his d^>artmsnt 
heads. ^ Lester Brown collaborates with Sam Holitiberg in setting 
instructional goals for the social studies departatient and insists that 
these goals are met. He also pays meticulous attention to the details 
of pedagogical practices, exam results, and achievement levels. 

Besides this enphasis on evaluation and his regular collaboration 
with the social studies department head, Lester Brown's image as an 
instructional leader is reinforced by his direct participation in and 
sometimes initiation of instructional iitprovement programs at 
Carlsberg. ^ Lester Brown occasionally teaches in the classroom, gives 
demonstration lessons ard leads workshops on instruction and pedagogy. 

If lester Brown at Carlsberg pays careful attention to the details 
of instruct iorial si5)ervision and evaluation, Scartoorou^'s Ken Olds 
f causes more on administrative matters ani coordination of district 
initiatives on thinking at Scarborou^ and elsev*ere in the Scarborou^ 
school system. Furthennore, \Aule his counterparts at Arnold, 
Carlsberg, Bradley, and Grandville are assigned to their respective 
buildings fulltiine. Ken Olds is 50% time at Scarborou^. He spends the 
other^ 50% of his time on his responsibilities to the district-wide 
training program on management skills for principals ana assistant 
principals in the school district. Ken Olds actively works at two 
jobs— one in vMdi he carries district initiatives to several schools 
arKi the other as the building principal at ScariDorou^. Olds' role is 
further conplicated by the fact that Scarborou^ is a predominantly 
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black schDcl vMcii is currently iinder court order to integrate. The 
school has attenpted to fulfill this requireitvent wj.thout using the 
e^qpedient of enforced busir>g. It has therefore created magnet 
programs, such as International Studies and Hicfa Tech ^ to attract hi^ 
achieving students who are mostly vdiite. In the last four years or so, 
the percentage of ^^Mte students in the school increased frcra 10% to 
32% (the percentage of blaclc students is now 66%) . Ifen Olds role as 
principal and the type of si^port he is able to provide for thinking at 
Scarborou^ must ttierefore be seen in the context of the school's 
desegregation efforts. 

Unlike Carlsberg, ^ere district monetary and technical siqpport for 
thinking is regarded as inadequate, Scarborouc^ is the beneficiary of 
relatively substantial district si:?:port for hi^er order thinking. Ihe 
district has in this sense, sou^t to prcawote this inner-city school 
(only 38% of Scarborou^'s students go on to college) as a model 
institution. Ken Olds describes the relationship between the district 
and school in the foil wing terms: "the cerAral office (the district) 
has a major iirpact on the school * s curriculvon and instructional 
practices. " This ""major iitpact" is reflected in the fact that 
Sca2±orou^ has the largest number of hi^er order thinking related 
programs and the hi^iest institutional profile for hic^ier order 
thinking of the five schools studied. 

Central to the instructional innovations being piloted at the 
school is the Scarborou^ Districtvade Critical Thinking Program 
(SDCTP). The program is funded jointly by the district and a large 
corporate benefactor to the tune of $300,000 over a two year period. 
The program involves 44% of Scarborou^ faculty. Teacher training and 
staff develcpnent in this program focus on curriculvan and instructional 
strategies for proamoting students "higher level thought pxx::esses." 
Errfiiasis is placed on divergent thinking and open-ended classroom 
discussion. To this end, a special interdisciplinary reader series. 
Response ^ has been developed for use in the classroom. The reader 
consists of short, often provocative passages on a variety of topics in 
philosc^y, aesthetics, history, etc., ^ch are intended to move 
teachers away from the traditional role of content dissemination and to 
stimulate student centered discussions. Another vital conponent of the 
SDCTP is the Syllabus Driven Exam an innovative approach to evaluation 
in social studies and other disciplines that aligns" district tests with 
district curriculim syllabi. The Syllabus Driven Exam places ercphasis 
on extended essay writing and is an atteiipt to provide an alternative 
to standardized or multiple choice tests. 

Programs such as SDCTP, magnet programs in arts and sciences, and a 
teacher center (established for district-wide teacher training) help to 
landerscore Scarfx^rou^'s status as a model public school in the 
district. But this status is a mixed blessing for the school and the 
principal. The cofftplicated internal demands of running a multiplicity 
of innovative programs along with his responsibilities to the distirict 
has meant that Ken olds has had to delegate much of his responsibility 
for day-to-day running of the school to his assistant principals and 
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other loenibers of staff. These substantied external and adrtiinistrative 
demands significantly reduce the ajDount of tim the principal can 
devote to curriculm and instruction mtters at Scarborou^. Ken Olds 
indicates that his involvement in instructional si:5)ervision has 
declined. "I have the additional responsibility this year of working 
as a resource to principals in other schools. Fifty percent of iry time 
is spent away frm the school. As a result, itty classroom observations 
have dropped considerably." 

\Mle Ken, Olds' involvement in the day-to-day instructional matters 
at Scarix>rou^ is limited, he reports that he uses his administrative 
leverage in the school to iaflioence desired instructional goals. This 
is done primarily by means of manipulation of resources 
("inducements") . In addition. Olds "encourages" his teaching staff to 
be self reliant.^ ^ He espouses a philosophy of "enpowerment" arid 
believes that decision making over curriculum and instructional issues 
should happen at the department level. But Sca2±>orDU^»s social 
studies d^>artment head, Mike Preminger, does not regard hineelf as an 
instructional leader. He defines his role within the department as 
that of an intermediary \*io gets his colleagues the curriculum 
materials and other resources they need for classroom instruction. In 
scane respects then, the philosophies of Scarborou^'s principal and 
social studies department head contradict rather than complement each 
other. While Olds claims to rely upon department-based instructional 
initiative, Preminger suggests that instructional leadership should 
come frcan the top. These different perspectives seem to have led to 
the fact^ that initiative and control over the hi^er order thinking 
instructional program have been assumed by the district. Rather than 
generating its own instructional agenda, the social studies department 
at Scarix)rou^ follows the district's approach to hi^er order 
thinking. 

District leadership for Scarborou^ has gone further than ai^ of 
the other schools in our saitple to institutionalize instructional 
programs for the promotion of hi^er order thinking. But the 
externally motivated institutional innovations seem initially to have 
diminished teacher control over the form and pacing of the 
instructional program. However, Preminger expressed some hope that 
following the pilot phase for the syllalbus driven courses, the nature 
of exam questions would be sufficiently specified and limited by the 
district so that teachers would have more tim and autoncatty to teach 
non-e3canined topics. 

For Martin Broderick, issues of hi^er order thinking, indeed 
issues of instructional improvement in general, have been secondary to 
an overvdielming concern with bringing "order" and "civility" to 
Bradley. Unlike, Scarborou^ there are no district motivated or school 
directed programs to promote hi^er order thinking instruction. The 
general approach with respect to instructional issues is to rely vipon 
department chairs to fine-tune the curriculum, but accortiing to both 
the principal and the social studies departnnent head, teachers at 
Bradley have a lot of autonomy. Althou^x Broderick has some ideas and 
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vague plans for hic^er order thirikirig, he had not devslqped them into 
programs, i^j^ooxding to the principal, it is the deporbnent heads, like 
iDuise Ellsworth in social studies, \^o should define the curriculum 
and instructional ageiida, Martin Broderick provides the administrative 
si:$ports and inducaments: 

Ihe primary instructional leadership responsibilities 
rest at departn^t head level, department heads define 
the agenda for the departments, the principal sij^ports 
these efforts by recommerxiing fiscal sij^port throu^ 
v?orkshqp monies and "days away" from school, etc, 
[Martin BrodericJc.] 

Broderick's enphasis on administration aid order has to do, he 
says, with the "near chaos" he inherited \*ien he arrived at Bradley 
about seven years ago: 

When I first came to Bradley, the school had a 

number of serious problems : overt and covert racism, 

violeiKie, vancJalism, graffiti, low achievement of 

blacks, lew teacher morale~rauch tension, kids 

leaving to attend private schools • Ttie main challenge 

was to bring order and stability, (M,BO 

The next iitportant issue for Broderick was the curriculum. At the 
tijne of his arrival, liberal education at Bradley was a "coaipletely 
elective curriculum". Althou^ he believes in faculty responsibility 
for the curriculum, he has been concemed with the fragmentation and 
chaos spawned by too many electives. Therefore, Martin Broderick has 
insisted cai some core course requirements for graduation and he sees 
his contribution to the curriculum as primarily one of imposing 
"structure" and "focus." Althou^ he virged a ti^tening of the 
graduation requirements, he did not immerse hiinself in specific issues 
of content within courses. 

Martin Broderick believes that the issues of school climate and the 
laissez-faire curriculum must both be addressed before the school can 
embark fully on a sitostantial program in instructioxicil innovation. For 
Broderick, Bradley is at a transitional stage with respect to hi^er 
order thinking, but he sees najor qpportunities for this beyond the 
classroom in the new system of sdiool governance he initiated. In the 
"tovm meeting," the students and faculty participate in decision making 
on a varied of issues such as class atten3ance, graduation 
requirements, and student behavior (eating, use of tape players, etc) . 
For Broderick, the tcwn meeting is an extracurricular ej^jerience in 
hitler order thinkii^ that allows students to particij^te in the 
governance of the school. Ihe town meeting is considered by the 
principal to be a forum in which the students can challenge each other 
and also their teachers over inpprtant ideas. Hai/ever, Broderick is 
less involved in pressing for higher order thinking in the classroom 
portion of the instructional program at Bradley. Like the principal at 
Scarborcu^, l^lartin Broderick adopts an indirect approach to 
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instructional iapravement in his school. He is far less involved in 
si:?)ervision and evaluation of his teaching staff than principals in the 
other schools, each of vAiom report visiting twice ^ many social 
studies teachers per semester. 

Despite his relatively limited contact with Bradl^'s teaciiers, 
Broderick supports Ellsworth's efforts in social studies, and he ranks 
the social studies department among the top departments at the school. 
Ellsworth, in turn, sees the principal as providing primarily iroral 
support, rather than specific intellectual leadership, for hi^ier order 
thinking initiatives at Bradley. Broderick views his contribution as 
providir^ the prerequisites for hi^er order thinkii>g, namely, in 
bringii^ a much needed cohereix:e and sense of focus to the curriculm 
and the liberal education program at Bradley; and in settir^ a school 
tone ("an orderly, disciplined academic atmosphere") for thinking. 

Like Broderick, the principal at GrancJville, Quentin Donato, also 
leaves the task of close instructional supervision to his department 
heads ("I do few dDseirvations") . However, he makes an ittpression as a 
systematic and methodical thiriker—with a firm and dear position on 
the kind of thinking that he believes must be promoted in the school, 
Grandville's principal offers a more sqphisticated conception of hi^er 
order thinking than any of the other principals — a conception vMch 
involves a critique of SLocm's taronomy ("Bloom's taxonomy is, I think, 
useful but only as a descriptive not prescriptive model,") • Quentin 
Donato r^rts that he systematically si^^ports thinking in the school 
and the social studies d^jartment throu^ the following methods: (a) 
promoting peer st5)ervision; (b) personnel selection and recruitment; 
(c) monies and resources for staff developtvent; and (d) eiKx>uraging an 
interdisciplinary focus. 

Two factors at Grandville reinforce Donato 's initiatives. First, 
the school climate is stable, and enriched by a hi^y cojtrmitted 
faculty and a tradition of academic achievement. Over 70% of 
Grandville "s students go to college — almost twice the percentage of 
students at Scarborouc^. Secondly, the school has a unique 
relationship with the district office in that the district's social 
studies coordinator is based at the school and leads the crusade for 
higher order thinking, 

Grandville 's sense of institutional stability results in part fran 
the relatively low rate of turnover of faculty and academic personnel. 
Biey have had only thi^e principals in the last 50 years. Donato notes 
that there is a "comnon college-oriented set of values among the 
students, staff, and parents." ohe prircipal has made a concerted 
effort to promote peer siqp^rvisiot) am a sense of collegiality and 
ccaranunity among faculty and ^^^ents. For exaitple, members of faculty 
are strongly encouraged to work with student organizations, and there 
is a student-faculty council at Grai^ville scHnewhat similar in spirit 
to the tcwn meeting idea at Bradley. Donato believes that 
extracurricular activit:ies such as the student-faculty council help to 
promote a climate of receptivity to thinkir^ amoi^ students. 
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Ihe princdpal gets solid siqoport frm the district office. Irdeed, 
the district office has established a definition of critical thinking 
and has itiitiated efforts through its subject coordinators to iitplement 
its agenda for curriculum and instruction in the schools ♦ Grandville 
benefits froro having such a subject coordinator, Bd3 MiKkelson, on 
staff in its social studies department. As v;e noted earlier, 
Grandville has develc^)ed a systematic approach to thinkii>g that is 
teacher produced and central to its instructional program, Donato and 
MiJdoelson collaborate in promoting the critical thirfdng program at 
Grandville. Ihe principal, for instance, indicates that the 
realizaticai of the critical thirikirg program in social studies at 
Grandville would have been "inconceivable" without MUdcelson. He has 
operated as both the architect as well as the chief practitioner of 
hitler order thinking instruction in the social studies department. 
Ihe district has provided significant inprovement nonies ($17,000) per 
year that provide s^joppart for outside consultants, staff development 
programs, peer observation, summer curriculum writing, and workshops 
related to hi^er order thinking. Grandville 's teachers have also been 
able to visit and leam from colleagues in other schools. 

It would seem that more than any school in our sanple, Grandville 
benefits frcan a fortuitous relationship with the district \ihidh is both 
si^portive but allows for a degree of autonony. Hujs type of district 
support and the collaborative efforts of the principal and department 
head have helped to sell the Grandville critical thlnkirg program to 
the teachers. 

Arnold does not enjoy the kiwa of direct district si^^port that is 
found at Grandville and Scarborou^. Both the principal, Paul Benton, 
and the^ department head. Herb Dexter, at Arnold criticize the district 
for failing to establish an initiative with respect to critical 
thinking. Paul Benton feels that the district has its priorities 
sonvsvAiat mi^laced: 

Ihere is no significant contribution or promotion 
[of hi^er ore? thinking] frm the district. 
District concerns are more mundane, more focused 
on test scoresand developing a consistent and 
district-wide curriculumat this point. Aside from 
funding one or twomentorships. . .There has been no 
direct district interest or awareness [P.B.I. 

Paul Benton also canplains that "administrative reorganization" in the 
district has significantly reduced the amount of time that he has to 
spend on curriculum and instructional ijtprwement ("Itoo much paper 
work") . 

Instructional momentum on thinking at Arnold is directly due to the 
support of the principal, and the unflagging efforts of his social 
studies department head. Dexter, and individual mentor teachers such as 
Bill Straj^om. Ihe relationship between Arnold's principal and the 
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social studies d^iartanent head is one of trust and routual respect. 
Paul Bentoi feels confortable in allowing Dexter and the social studies 
department to have considerable autoncny. He does not feel that it is 
necessary for him to be directly involved in the social studies 
d^artment "because the department is by far the best department in the 
school." Dexter, in turn, regards Benton's moral support and guidance 
as crucial to instructional initiatives in hi^x order thinkina at 
Arnold. 

Arnold's principal sees himself as a curriculum and instructional 
leader. . ."sonething of a task master." He takes pride in this image of 
himself. "I work very hard and I ej^ject ray staff to work hard...! 
expect performance and results." Paul Benton, like Lester Brown at 
Carlsberg, is an instructional leader as well as an active 
administrator vdio is keenly in touch with the daily operations of his 
sciiool. He spends a lot of time talking and workii^ with teachers to 
get a better understanding of their needs. He pays special attention 
to new staff members vAio "need help in the classroom." Paul Bentcai 
shares responsibility for protroting effective teachii^ with his 
d^artment heads and ei^t senior/mentor teachers at the school, ihese 
mentor teachers are ftmded by the district. At Arnold, they work 
closely with other members of the staff to help inprove the overall 
quality of instruction. With the principal's urging, mentor teachers 
also hold workshcjs and inplement innovative curriculm projects. 
Arnold's writing project, led by Bill strajiiom vMch ^icludes a strong 
thinking ccatponent and involves an interdisciplinary collaboration 
between the English and Social Studies deparbnents is a prime example 
of the mentor initiative. 

In addition to his active prcanotion of instructional improvement, 
Paul Benton deliberately vises personnel selection, recniitroent, and 
cultivation of leadership to achieve his goals. Accordir^ to Benton, 
when he arrived at Arnold about six years ago, "the average staff 
member was fifty-four, tired and scaling down." Benton set about 
revitalizing the staff, and within the last three years, there has been 
a 30% turnover. Benton set about revitalizing the staff. He used 
'•every opportunity to hire quality teachers interested in instructional 
inprovement and taking on diEillenges in the classrocsn." 

Closely related to his enphasis on personnel selection, the 
principal has worked actively to "cultivate instructional leaders" 
among Arnold's teachers and youth leaders aitcng the student body. 
Benton regards this "quality circle" of teadiers and students as 
"partners" in his efforts to develop a manentum for thiriidng at Arnold. 

m. 

Ocndusicns 

What, if anything, can we say about the iiipact of the department 
head's and the principal's leadership on the level of thinkii^ actually 
found in the dassrocm? 
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let V3 begin with a necessarily cursory overview of our f iixiings at 
the classrooa level. A total of 165 social studies classes were 
observed by one or two observers. Each class was rated along 14 
different 5-point scales, of vMch six were picked as constitutive of 
Hitler Order Blinking, ihe numbers belcw in parentheses r^resent an 
average of these six different scales. Five is the hi^iest possible 
score and one^ the lowest. 



Ohe hi^iest scoring school was Carlsberg (4.13). Next came 
Grandville (3.99) / then Arnold (3.91), Bradley (3.50) and finally 
Scarborou^ (3.15). Ihese ratings are not siirply reflections of the 
individual teachers in the five s<diools. Diffei^ences between schot?ls 
account for 28% of *bhe total variance. The hic^^st school differs from 
the lowest by 1.70 standard deviations, and frcxa the fourth-place 
school by .93 standard deviations, ttius^ institutional effects are 
inportant. Furthermore^ the scores are not jvist reflections of the 
socioeconcfftdc or racial costposition of the five student bodies. For 
exanple, only half the student body at Carlsberg is vMte ccnpared to 
75% at Bradley. let us now consider a number of plausible propositions 
about institutional effects and see how our data speak to tSem. 

1. A systematic^ department-based program/ focused on hi^ier^rder 
thinking is necessary for obtaining classrooms which score hi^ in this 
area. 

3he evidence frcan our studj^'' si^ports this. Ihe three ni^er- 
scoring schools stand out in this respect althou^ there are 
differences in the way in \Mch they concentrate on hi^ier-order 
thinking. Both Arnold and Grandville try to transmit a particular 
conception of HOT/ whereas Carlberg considers the stimulation of 
student thinkii^ as just one facet of pedagogical ejccellence. 
Scarborou^/ althouc^ it has a school-wide program/ has no systematic 
effort at the deparbtental level. Finally/ Bradley has no systemtic 
program in the area. 

• 

2. Success in this enterprise requires substantial resources/ 
inclvding personnel/ set aside for this purpo^/^e. 

Ihis is not supported. Grandville haS/ for exaiiplG/ allocated 
substantial resources into its hi^er order thinking program while 
Arnold has virtually Done, yet their scores are about the same, of 
cwrse, the fact that Arnold can succeed with limited resources 
provides no guarantee that other schools can. 

3. Success in this area depends on stroi^ leadership at the departnisnt 
level. 

Ciis notion receives strong support. ohe department heads at 
Carlsberg/ - Grandville/ and Arnold are dynamic leaders who ia^spire 
conmitjnsnt. Ohey work energetically to improve the quality of thinkir^ 
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in their departitient's cdasses. Bie departinent heads at Bradley and 
Scarborou^ differ in many ways, but both take a relatively non- 
directive stance vdien it comes to shaping their colleagues* teacihing 
stales in particular directions. 

4. Ihe principal must play an active role in siqpporting the d^>artment 
head's efforts. 

OSiis notion receives strong si?>port. In all three of the top 
schools, principals were very active in working closely with department 
heads, but in neither of the bottom two did this occur. 

5. District-wde efforts will be in vain if the teachers and their 
immediate supervisors, the department chairs, do not feel "ownership" 
of the program. 

This notion, derived from so much recent literature on the subject 
(e.g., Stevenson, 1987), is clearly siqpported in the two schools that 
have district-wide programs. Grandville and Sc^rborou^ present the 
clearest contrast. In one case (Grandville) , the initiative has cane 
from the person viio was also the department head in a single hi^ 
school district; in the other case, the departiaent head of a multi- 
school district appears to be left out of any of the school-mde 
efforts. 

6. Success depends upon an institutional culture of oollegiality vMch 
involves consistent, focused discussion of teaching and curriculum 
within the department. 

Ihis is confirmed: the top three but not the bottom two schools 
may be accurately characterized in this way. 

Our study reinforces vAiat many others have found, that the 
institutional context does make a difference. Ooll^ial school culture 
and pedagogical leadership from department heads and principals help 
teachers promote hi^er order thinking in their dassrxxsns. Ihere do 
not appear to be sinple, administrative actions vMch can be oaanted on 
to produce the culture that raises the level of thinking in classrooms. 
Whatever actions may be needed, our stud^ irjdicates that energetic 
instructional leadership at the building level plays a significant 
role. 
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Tred M. Neunann 

In spite of numsrous prcposals that thirikiiig ou^t to be 
€n?iiasized nore in sdiools/ stiriies heretofore have paid little 
attention to such basic questions as 

To vfiiat extent can hi^ier order thinking actually find a prcaninent 
place in hi^ schools that are conventionally organized? 

Why is it so difficult to promote? 

Why have sane schools made successful laoverrent toward this goal, 
and mat have they done to overccatae soaae of the barriers? 

Answering these questions involves a xiumber of steps, sane of vMch 
have been ooiplefced in this stud^r, but others await future research. 
Rather than surarrarizii^ each of the papers separately, I will atteitfrt: 
to synthesize the findings as they pertain to these issues. 

We have made progress on the first question by developing a 
conception of hi^ier order thiiiking applicable to teachiirf a variety of 
content and by operationalizing this conception into a reliable schene 
for classrocaoa dDservation. By studying a group of five departinents 
selected for their special efforts to proaiote hic^er order thinking, we 
may have also obtained estimates of the "i:55)er limits" of 
thou^tfulness in conventionally organized social studies departments. 
Whether the le-'aLs of discourse we found are truly exeitplary will not 
be determined until we study a more r^resentative set of d^)artraants 
in the next phase of the research, but at this point have evidence 
that sane departments demonstrate in their lessons hi^ levels on the 
following dimensions: 

sustained examination of a fm topics; 

substantive coherence and continuity; 

sufficient time for students to respond; 

the posing of challenging, non-routine intellectual tasks; 

teacher modeling of thou^tful problem solving behavior; 

students offering explanations for their conclusions. 

In contrast, over all lessons we found less evidence of several 
dimensions of thou^t fulness: teacihers' carefial consideration of 
e}5)lanatiQns and reasons, Socratic questioning, integrating students' 
personal e5$>erience into the lesson, encouraging students to come vp 
with original ideas, que^ioning authoritative sources. Althoui^ such 
dimensions were relatively rare across all lessons, the hi^iest scoring 
lessons, vdien ccaanpared with the lowest, did manifest more teacdier 
careful consideration of reasons and Socratic questioning, and their 
students more often generated original ideas and articulate, relevant 
canments* Biese classes also relied more on class discussion, rather 
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than lec±ure, and the use of sources other than textbooks. While there 
was considerable variation in thou^tftilness both within and between 
schools, we found scxdb inpressive exarrples of students being diallenged 
to lase their minds. 

In addition to researchers' observations of cognitive cSiallenge in 
classes, the vast majority of students told vb about being challenged 
and eng&ged in certain classes. Students ncxninated social studies more 
frequently than any other subject as the most dicillenging class taken 
in the present school year and also thrcoc^iout hi^ school. And most 
students indicated that the class that was most challenging also was 
their most engaging class. IJhen asked to describe the Jdnd of social , 
studies work th^ consictered loost challenging, stidents identified such 
tasks as forming one's opiniai, making inferences, and other tasks 
consistent with our definition of hi^^ order ciiallenge. 

Teachers offered information on barriers to prcraoting hi^ier order 
thinking. Across all sdiools, teachers believe that large class size, 
large nunibers of classes per day, and large student load restrict their 
opportunity to promote thinking. Ihere is also wide agreenent that the 
pressure to cover content, whether externally- or self-iitposed, often 
inhibits thinking. Teachers occasionally ooatrplained about lack of 
appropriate instructional materials and mandated tests, but these were 
not prominent barriers. Teachers in all the schools generally found 
adequate collegial arxi administrative si^^port for an enophasis on 
thiridng. 

Student ability or background did not appear as a barrier to 
classroom thou^tfulness in these schools. In the interviews teachers 
rarely attributed problems in promoting thinking to student 
characteristics. Quantitative analysis of lesson scores shewed only 
minor effects of student background, and the teachers vdio scored the 
hi^iest tau^t more low achieving students than did their lower scoring 
colleagues. 

Cocparisons between the hi^ and lower scoring teachers suggested 
that important barriers may reside in teachers' goals and oonc^ions 
of their work. Periiaps not surprisingJy, teadiers with the most 
thou^tful classes placed hi^er priority on thinking as an educational • 
goal, articulated more elaborate ooncqptions of thinking, and placed 
more value on depth than breadth of coverage. Ihis rcdses, but cannot 
answer, the question of the extent to which teacher beliefs must be < 
considered the central target for reform in this area, in contrast, for 
exanple, to materials and organizational conditions. 

Ifow mi^t we esqplain differences in classroom thou^tfulness 
between sdiools? VBiile most variation in tha^tfUlness is due to 
differences between teachers, more than 25 percent of the variation can 
be attributed to schools. The five schools differ in iirportant ways, 
and the small saiiple wHJ not allow powerful generalization, but we 
found si5:port for some propositions that distinguish the top three from 
the bottom two schools. 
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Each of the top three schools, but neither of the bottaa two, had 
sanoe systematic, dqpartinentally-based program that si?:ported hi^ier 
order thinking. Bie content of the "programs" differed substantially: 
one followed a nationally prcmsted scheme of convergent thinking 
skills; another eirphasized a standard lesson format that began with a 
lesson edm stated as a prcablem or question and looved ftan informational 
to evalixative levels of analysis; the third placed inajor enophasis on 
divergent thinking and visual representation of thinking throu^ 
metaphors and diagirams. Ihe lowest scoring school was part of an 
extensive district-^de program, but here there was little sense of 
d^jartmental ownership. Ihe top three schools also manifested a 
d^>artmental culture of collegial planning absent in the bottom two, 
and they similarly distinguished themselves throuc^ vigorous 
instructional leadership aimed at hi^er order thinking by both the 
d^)artment dhair and the principal. 

^ Together these findings indicate that classroom thou^tfUlness in 
social studies can be assessed and that it can occur at hi^ levels 
among a critical mass of teachers in conventionally organized hi^ 
schools and with classes ccarposed of students of all grade and 
achievement levels. We have also found that students" consider 
thcu^tful classes to be the most challenging and ei^ging, and we have 
identified the particular attributes that students mention in making 
such judgmentSo 

In spite of this good rmis, we find \*at appear to be irrportant 
differences in thinking between teacihers v4io prcccte hi^ier order 
thinking most and least consistently. Althou^^ we cannot point to 
particular administrative actions vMcii guarantee the proaoaotion of 
thinking in social studies d^)artment£i, it seems that departmentally- 
based programs, collegial faculty culture ana strong leadership by the 
chair and principal may be critical. 

Still, there is much to learn. We have ix)t investigated the 
relationship between classroom thou^tfulness and individual student 
achievement. Neither have we tested the extent to vMch these five 
select d^)a2±oaents actiaally proonnote hi^^er levels of thoui^tfulness 
than a more representative set. Finally, eathou^ we may find hi^ 
levels of thou^tfulness in some conventionally organized hi^ schools, 
concerns about organizational barriers expressed by teachers and 
reformers invites inquiry into a third set of departments - those with 
structures modified moire deliberately to promote thou^tfulness. 
IXiring the next two years, each of these problems will be studied. 
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